Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Simple tasks showing reasoning breakdown in state-of-the-art LLMs

arxiv.org

Post reply on HN
Print view View original

111–120 of 393 posts

Re: Simple tasks showing reasoning breakdown in state-of-the-art LLMs

#111

Post by camdenreslink »

tsunamifury wrote: post #90
pawelmurias wrote: post #59

Earlier quoted context omitted.

LLMs are idiots. They can't reason properly and only parrot stuff https://chatgpt.com/share/dcb4ff4e-e8a2-463b-86ec-9caf10b6e6... Sometimes they get the answer right to something really complex because it fits a pattern, but sometimes they answer with something really really stupid.

Why are so many people so insistent on saying this? I’m guessing you are in denial that we can make a simulated reasoning machine?

It is hard to trust any output from a machine that is confidently wrong so frequently. You need to already be knowledgable in a topic (or at least have a well attuned BS detector) to know if it is giving you correct responses. It can be a time saver and assistant in getting work done where you are already a subject matter expert, but it needs to get better to remove the human from the loop.

Re: Simple tasks showing reasoning breakdown in state-of-the-art LLMs

#112

Post by Karellen »

GPerson wrote: post #10
Closi wrote: post #3

Question is: "Alice has 60 brothers and she also has 212 sisters. How many sisters does Alice’s brother have?" (nb: I have added numbers, it's phrased as X and N in the paper) I must confess, when I tried to answer the question I got it wrong...! (I feel silly). I only realised I got it wrong when I plugged it into GPT-4o and it came back with the correct answer: https://chatgpt.com/share/6eb5fa36-e0fd-4417-87d1-64ca…

There must be a name for the new phenomenon, of which your post is an example, of: 1. Someone expresses that an LLM cannot do some trivial task. 2. Another person declares that they cannot do the task, thereby defending the legitimacy of the LLM. As a side note, I cannot believe that the average person who can navigate to a chatgpt prompter would fail to correctly answer this question given sufficient motivation to d…

You could call it the "Compose a symphony fallacy", after the scene in I, Robot (2004) where Spooner asks Sonny if he can compose a symphony, to which Sonny replies "Can you?"

Re: Simple tasks showing reasoning breakdown in state-of-the-art LLMs

#113

Post by ip26 »

nerdjon wrote: post #54
photon_lines wrote: post #44

Earlier quoted context omitted.

There's actually a pretty simple solution to this that I thought about testing out and it involves asking the model to re-construct the problem using a logic language (like Prolog) and asking it to execute this type of program in order to come up with a solution rather than attempting simple chain-of-reason training / other methodologies of getting the model to 'reason' through some of these examples. People forget t…

I honestly thought about this recently when I was trying to see the limits of Claude Opus. Some of the problems I gave it, what if instead of telling it to solve the problem I asked it to write the script and then give me the command and inputs needed to properly run it to get the answer I needed. That way instead of relying on the LLM to do properly analysis of the numbers it just needs to understand enough to write…

I don’t understand why LLM’s aren’t already set up to do what you describe automatically behind the curtain. Extract a math equation from text (LLMs are good at translating between languages right?) and immediately evaluate it on the host CPU.

LLM is the equivalent of recalling your times tables. Computer arithmetic is the equivalent of re-computing your times tables.

Re: Simple tasks showing reasoning breakdown in state-of-the-art LLMs

#114

Post by ticklemyelmo »

nottorp wrote: post #43

Of course, the initial mistake is assuming LLMs have any 'reasoning'...

"The breakdown is dramatic, as models also express strong overconfidence in their wrong solutions, while providing often non-sensical "reasoning"-like explanations akin to confabulations to justify and backup the validity of their clearly failed responses, making them sound plausible."

It's fascinating how much they anthropomorphize the systems and credit them with emotional, possibly deceitful behaviour, in a paper trying to explain how unintelligent they are.

Re: Simple tasks showing reasoning breakdown in state-of-the-art LLMs

#115

Post by dr_dshiv »

colechristensen wrote: post #40
nerdjon wrote: post #18

For anyone considering reading the paper and like me don't normally read papers like this, open the PDF and think you don't have time to read it due to its length. The main part of the paper is the first 10 pages and a fairly quick read. On to the topic here. This is an interesting example that they are using. It is fairly simplistic to understand as a human (even if we may be inclined to quickly jump to the wrong co…

I don't think anybody who actually uses LLMs thinks they are general purpose AI. Like you ask it to do one thing it's amazing, but then you try to modify or do something with extra steps, or just anything with any complexity to it and it falls over.

Well, I believe LLMs are general purpose AI. And, for the record, so does Peter Norvig. He coauthored the most popular textbook on Artificial Intelligence, so this belief can be considered a somewhat mainstream perspective. His article, “Artificial General Intelligence Is Already Here“ [1] is really worth reading.

https://www.noemamag.com/artificial-general-intelligence-is-...

Re: Simple tasks showing reasoning breakdown in state-of-the-art LLMs

#116

Post by CooCooCaCha »

sollewitt wrote: post #105
layer8 wrote: post #94

Earlier quoted context omitted.

For reasoning of any complexity, some sort of internal monologue and iteration is needed. For type-2 thinking, we conceptualize possible solutions, arguments, and reasoning paths in our mind, and judge and evaluate them before deciding which one we think is adequate or correct. This can be emulated to some extent by guiding prompts that make that internal thought process external, but we really want it to be internal…

Given many people don’t have an inner monologue and function just fine, it’s more likely inner monologue is a product of the reasoning process and not it’s mechanism.

I think you’re using “inner monologue” too literally. It could be a progression of pictures, emotions, etc.

Re: Simple tasks showing reasoning breakdown in state-of-the-art LLMs

#118

Post by ericmcer »

nerdjon wrote: post #18

For anyone considering reading the paper and like me don't normally read papers like this, open the PDF and think you don't have time to read it due to its length. The main part of the paper is the first 10 pages and a fairly quick read. On to the topic here. This is an interesting example that they are using. It is fairly simplistic to understand as a human (even if we may be inclined to quickly jump to the wrong co…

If you really think about what an LLM is you would think there is no way that leads to general purpose AI. At the same time though they are already doing way more than we thought they could.

Maybe people were surprised by what OpenAI achieved so now they are all just praying that with enough compute and the right model AGI will emerge.

Re: Simple tasks showing reasoning breakdown in state-of-the-art LLMs

#119

Post by layer8 »

sollewitt wrote: post #105
layer8 wrote: post #94

Earlier quoted context omitted.

For reasoning of any complexity, some sort of internal monologue and iteration is needed. For type-2 thinking, we conceptualize possible solutions, arguments, and reasoning paths in our mind, and judge and evaluate them before deciding which one we think is adequate or correct. This can be emulated to some extent by guiding prompts that make that internal thought process external, but we really want it to be internal…

Given many people don’t have an inner monologue and function just fine, it’s more likely inner monologue is a product of the reasoning process and not it’s mechanism.

It’s commonly conjectured that the emergence of human-level reasoning wouldn’t have been possible without the development of language. Personally, I’m able to suppress “word thoughts” in my head (for a short time), but then I lose almost all of my reasoning ability. I could imagine that reasoning is language-based even when it’s not conscious for some people. An internal process being there, and being conscious of it, are two separate things. We would be happy with an AI using an internal monologue without it being conscious of that monologue.
Post reply on HN