Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Simple tasks showing reasoning breakdown in state-of-the-art LLMs

arxiv.org

Post reply on HN
Print view View original

101–110 of 393 posts

Re: Simple tasks showing reasoning breakdown in state-of-the-art LLMs

#101

Post by hyperbovine »

tsunamifury wrote: post #90
pawelmurias wrote: post #59

Earlier quoted context omitted.

LLMs are idiots. They can't reason properly and only parrot stuff https://chatgpt.com/share/dcb4ff4e-e8a2-463b-86ec-9caf10b6e6... Sometimes they get the answer right to something really complex because it fits a pattern, but sometimes they answer with something really really stupid.

Why are so many people so insistent on saying this? I’m guessing you are in denial that we can make a simulated reasoning machine?

Because they understand how LLMs work. It's not reasoning. It's not simulating reasoning.

Re: Simple tasks showing reasoning breakdown in state-of-the-art LLMs

#102

Post by PheonixPharts »

tsunamifury wrote: post #90
pawelmurias wrote: post #59

Earlier quoted context omitted.

LLMs are idiots. They can't reason properly and only parrot stuff https://chatgpt.com/share/dcb4ff4e-e8a2-463b-86ec-9caf10b6e6... Sometimes they get the answer right to something really complex because it fits a pattern, but sometimes they answer with something really really stupid.

Why are so many people so insistent on saying this? I’m guessing you are in denial that we can make a simulated reasoning machine?

People keep saying it because that's literally how LLMs work. They run Montecarlo sampling over a very impressive latent linguistic space. These models are not fundamentally different than the Markov chains of yore except that these latent representations are incredibly powerful.

We haven't even started to approach the largest problem which is moving beyond what is essentially a greedy token level search of this linguistic space. That is, we can't really pick an output that maximized the likelihood of the entire sequence, rather we're simply maximizing the likelihood of each part of the sequence.

LLMs are not reasoning machines. They are basically semantic compression machines with a build in search feature.

Re: Simple tasks showing reasoning breakdown in state-of-the-art LLMs

#103

Post by mupuff1234 »

colechristensen wrote: post #40
nerdjon wrote: post #18

For anyone considering reading the paper and like me don't normally read papers like this, open the PDF and think you don't have time to read it due to its length. The main part of the paper is the first 10 pages and a fairly quick read. On to the topic here. This is an interesting example that they are using. It is fairly simplistic to understand as a human (even if we may be inclined to quickly jump to the wrong co…

I don't think anybody who actually uses LLMs thinks they are general purpose AI. Like you ask it to do one thing it's amazing, but then you try to modify or do something with extra steps, or just anything with any complexity to it and it falls over.

They might not think that but they sure as hell are trying to sell that idea.

Re: Simple tasks showing reasoning breakdown in state-of-the-art LLMs

#104

Post by hyperbovine »

waynecochran wrote: post #35

Citation 40 is the longest list of authors I have ever seen. That is one way to help all your friends get tenure.

Tenure committees have to write a report detailing every single one of your papers and what your contribution was.

Re: Simple tasks showing reasoning breakdown in state-of-the-art LLMs

#105

Post by sollewitt »

layer8 wrote: post #94
nerdjon wrote: post #18

For anyone considering reading the paper and like me don't normally read papers like this, open the PDF and think you don't have time to read it due to its length. The main part of the paper is the first 10 pages and a fairly quick read. On to the topic here. This is an interesting example that they are using. It is fairly simplistic to understand as a human (even if we may be inclined to quickly jump to the wrong co…

For reasoning of any complexity, some sort of internal monologue and iteration is needed. For type-2 thinking, we conceptualize possible solutions, arguments, and reasoning paths in our mind, and judge and evaluate them before deciding which one we think is adequate or correct. This can be emulated to some extent by guiding prompts that make that internal thought process external, but we really want it to be internal…

Given many people don’t have an inner monologue and function just fine, it’s more likely inner monologue is a product of the reasoning process and not it’s mechanism.

Re: Simple tasks showing reasoning breakdown in state-of-the-art LLMs

#106

Post by yoyohello13 »

GPerson wrote: post #10
Closi wrote: post #3

Question is: "Alice has 60 brothers and she also has 212 sisters. How many sisters does Alice’s brother have?" (nb: I have added numbers, it's phrased as X and N in the paper) I must confess, when I tried to answer the question I got it wrong...! (I feel silly). I only realised I got it wrong when I plugged it into GPT-4o and it came back with the correct answer: https://chatgpt.com/share/6eb5fa36-e0fd-4417-87d1-64ca…

There must be a name for the new phenomenon, of which your post is an example, of: 1. Someone expresses that an LLM cannot do some trivial task. 2. Another person declares that they cannot do the task, thereby defending the legitimacy of the LLM. As a side note, I cannot believe that the average person who can navigate to a chatgpt prompter would fail to correctly answer this question given sufficient motivation to d…

Many people, especially on this site, really want LLMs to be everything the hype train says and more. Some have literally staked their future on it so they get defensive when people bring up that maybe LLMs aren’t a replacement for human cognition.

The number of times I’ve heard “but did you try model X” or “humans hallucinate too” or “but LLMs don’t get sleep or get sick” is hilarious.

Re: Simple tasks showing reasoning breakdown in state-of-the-art LLMs

#107

Post by cma »

michaelfeathers wrote: post #82

This is a good talk about the problem: https://youtu.be/hGXhFa3gzBs?si=15IJsTQLsyDvBFnr Key takeaway, LLMs are abysmal at planning and reasoning. You can give them the rules of planning task and ask them for a result but, in large part, the correctness of their logic (when it occurs) depends upon additional semantic information rather then just the abstract rules. They showed this by mapping nouns to a completely dif…

People also fall apart on things like statistical reasoning if you switch domains (I think it is the Leda Cosmides evo psych stuff that goes into it but there might be a more famous experiment).

Re: Simple tasks showing reasoning breakdown in state-of-the-art LLMs

#109

Post by sollewitt »

photon_lines wrote: post #44
nerdjon wrote: post #18

For anyone considering reading the paper and like me don't normally read papers like this, open the PDF and think you don't have time to read it due to its length. The main part of the paper is the first 10 pages and a fairly quick read. On to the topic here. This is an interesting example that they are using. It is fairly simplistic to understand as a human (even if we may be inclined to quickly jump to the wrong co…

There's actually a pretty simple solution to this that I thought about testing out and it involves asking the model to re-construct the problem using a logic language (like Prolog) and asking it to execute this type of program in order to come up with a solution rather than attempting simple chain-of-reason training / other methodologies of getting the model to 'reason' through some of these examples. People forget t…

Right, and do you verify the result?

You have to know what the answer is supposed to be before you can write a test case.

Re: Simple tasks showing reasoning breakdown in state-of-the-art LLMs

#110

Post by throw46365 »

mjburgess wrote: post #33
nerdjon wrote: post #18

For anyone considering reading the paper and like me don't normally read papers like this, open the PDF and think you don't have time to read it due to its length. The main part of the paper is the first 10 pages and a fairly quick read. On to the topic here. This is an interesting example that they are using. It is fairly simplistic to understand as a human (even if we may be inclined to quickly jump to the wrong co…

In many ways, this is very obvious and routine to people who use these systems with a critical understanding of how they work. It's dispiriting how we require a arxiv.org pdf in order to point out the absence of the emperor's clothing. There are few people who use LLMs who could not, right now, demonstrate this point to themselves if they so wish. Such is the strength of corporate tech propaganda that a whole mass of…

> Such is the strength of corporate tech propaganda that a whole mass of people will instead insist that we have never worn clothes either.

This is the line of reasoning I find most dispiriting. I still believe tech people cling to this line of reasoning because it helps them justify replacing people in jobs with LLMs.

Post reply on HN