Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Simple tasks showing reasoning breakdown in state-of-the-art LLMs

arxiv.org

Post reply on HN
Print view View original

131–140 of 393 posts

Re: Simple tasks showing reasoning breakdown in state-of-the-art LLMs

#131

Post by glitchc »

nerdjon wrote: post #18

For anyone considering reading the paper and like me don't normally read papers like this, open the PDF and think you don't have time to read it due to its length. The main part of the paper is the first 10 pages and a fairly quick read. On to the topic here. This is an interesting example that they are using. It is fairly simplistic to understand as a human (even if we may be inclined to quickly jump to the wrong co…

The problem is a good chunk of the global population is also not reasoning and thinking in any sense of the word. Logical reasoning is a higher order skill that often requires formal training. It's not a natural ability for human beings.

Re: Simple tasks showing reasoning breakdown in state-of-the-art LLMs

#132

Post by glitchc »

ElevenLathe wrote: post #129
sollewitt wrote: post #105

Earlier quoted context omitted.

Given many people don’t have an inner monologue and function just fine, it’s more likely inner monologue is a product of the reasoning process and not it’s mechanism.

The existence of an "inner monologue" isn't really a falsifiable claim. Some people claim to have one while other people claim not to, but we can't test the truth of these claims.

In this particular case, is there any reason why we simply can't take their word for it? This is not a case of where if I say "weak" or "strong", most people pick strong because no one wants to be weak, even if the context is unknown (nuclear force for example).

Re: Simple tasks showing reasoning breakdown in state-of-the-art LLMs

#133

Post by kalkin »

> Paper finds a failure case for current LLMs and draws broad conclusions about the fundamental limits of LLM technology

> Evaluates the largest LLMs and finds evidence that actually scale overcomes the problem:

"Notable exceptions are Claude 3 Opus and GPT-4 that occasionally manage to provide correct responses backed up with correct reasoning as evident in structured step by step explanations those models deliver together with solution"

> Drink!

I'm not sure it's productive to be this sarcastic on HN, but it's really quite a common pattern. And there's something very frustrating about how authors of these papers will accuse others of hype and overstating results but also often vastly overstate the generality of their own results - to the point where this HN thread is full of people saying bluntly "this proves LLMs can't reason".

Re: Simple tasks showing reasoning breakdown in state-of-the-art LLMs

#134

Post by Mathnerd314 »

They repeat over and over that this is a simple task, but then they explain that they developed it by looking at math olympiad problems for young children ages 7-10. So it would be very interesting to know the performance of these elementary school children on the problem. My guess is that if it was on a math test they would also get confused and try to do random arithmetic.

Re: Simple tasks showing reasoning breakdown in state-of-the-art LLMs

#135

Post by photon_lines »

sollewitt wrote: post #109
photon_lines wrote: post #44

Earlier quoted context omitted.

There's actually a pretty simple solution to this that I thought about testing out and it involves asking the model to re-construct the problem using a logic language (like Prolog) and asking it to execute this type of program in order to come up with a solution rather than attempting simple chain-of-reason training / other methodologies of getting the model to 'reason' through some of these examples. People forget t…

Right, and do you verify the result? You have to know what the answer is supposed to be before you can write a test case.

Yup - well you feed in the prompt along with an answer and you get the model to produce outputs and check for discrepancies. If the answer is wrong then the model adjusts -- this is the way backpropagation works....I think there are huge advantages in using logic languages in order to represent some of these data sets rather than simple English or the current chain-of-thought reasoning approaches -- backpropagation as an example isn't really used in the human brain, but it leads to great results in mimicking how neural networks 'learn' - in the same way, we don't have to have the full formal picture of how humans model the logical world in order to achieve great results. We can simulate this using logic programming or even general programming or at least that's my conjecture.

Re: Simple tasks showing reasoning breakdown in state-of-the-art LLMs

#136

Post by Mmskynettio »

nerdjon wrote: post #18

For anyone considering reading the paper and like me don't normally read papers like this, open the PDF and think you don't have time to read it due to its length. The main part of the paper is the first 10 pages and a fairly quick read. On to the topic here. This is an interesting example that they are using. It is fairly simplistic to understand as a human (even if we may be inclined to quickly jump to the wrong co…

Its not an AI hype. A hype is defined as something which gets oversold: "promote or publicize (a product or idea) intensively, often exaggerating its benefits."

Just yesterday I visited a google cloud summit and one person from bosch told the audiance how they are now able to work with less external agencies like texting, graphicsdesigner and photographers for their materials.

It already saves money, has real impacts and continues to progress.

We are also don't know what ChatGPT 5 will bring, because they say this will do more reasoning than before, but we already are working (people/our socity) on solving this in different ways: From code which creates a unit test first and than the code, to different type of architectures.

For me, 2024 was the LLM cost reduction year and the LLM gets a big context window year.

AI doesn't need to be ready tomorrow, but its capabilities are already really good. And i know plenty of people around me who are a lot less interesting to talk to than any llm (from a human skill/knowledge point of view).

llama 3 was also a big achievement 2024. Facebook shows that better data leads to better quality for smaller models.

We haven't not only entered the AI ara but also the 'gather all the knowledge we can, quality check it and refine it because now we can actually do something with it' ara.

We are in the feedbackloop knowledge ara.

Re: Simple tasks showing reasoning breakdown in state-of-the-art LLMs

#137

Post by imtringued »

tsunamifury wrote: post #90
pawelmurias wrote: post #59

Earlier quoted context omitted.

LLMs are idiots. They can't reason properly and only parrot stuff https://chatgpt.com/share/dcb4ff4e-e8a2-463b-86ec-9caf10b6e6... Sometimes they get the answer right to something really complex because it fits a pattern, but sometimes they answer with something really really stupid.

Why are so many people so insistent on saying this? I’m guessing you are in denial that we can make a simulated reasoning machine?

No it is because supervised and self supervised learning happen to produce reasoning as a byproduct. For some reason people think that telling a model to recite a trillion tokens somehow will improve it beyond the recitation of those tokens. I mean, in theory you can select the training data so that it will learn what you want, but then again you are limited to what you taught it directly.

The problem is that these models weren't trained to reason. For the task of reasoning, they are overfitting to the dataset. If you want a machine to reason, then build and train it to reason, don't train it to do something else and then expect it to do the thing you didn't train it for.

Re: Simple tasks showing reasoning breakdown in state-of-the-art LLMs

#139

Post by solidasparagus »

colechristensen wrote: post #40
nerdjon wrote: post #18

For anyone considering reading the paper and like me don't normally read papers like this, open the PDF and think you don't have time to read it due to its length. The main part of the paper is the first 10 pages and a fairly quick read. On to the topic here. This is an interesting example that they are using. It is fairly simplistic to understand as a human (even if we may be inclined to quickly jump to the wrong co…

I don't think anybody who actually uses LLMs thinks they are general purpose AI. Like you ask it to do one thing it's amazing, but then you try to modify or do something with extra steps, or just anything with any complexity to it and it falls over.

The vast majority of people use a model built by someone else through a high-level abstraction and then make broad claims about what the technology will be capable of.

Then you have the people who are leveraging the technology to train models from scratch and that population is far more apt to believe that large models can be general purpose AI (by some definition). Sure there are other things at play like money, publicity, reputation, and a desire to do something important. But there is also the very clear trend line where transformers are able to model pretty much any sequence of tokens where there is pretraining scale data available. Whisper was not a surprise. GPT-4v was not a surprise. Sora was not a surprise.

Re: Simple tasks showing reasoning breakdown in state-of-the-art LLMs

#140

Post by [deleted] »

cma wrote: post #107
michaelfeathers wrote: post #82

This is a good talk about the problem: https://youtu.be/hGXhFa3gzBs?si=15IJsTQLsyDvBFnr Key takeaway, LLMs are abysmal at planning and reasoning. You can give them the rules of planning task and ask them for a result but, in large part, the correctness of their logic (when it occurs) depends upon additional semantic information rather then just the abstract rules. They showed this by mapping nouns to a completely dif…

People also fall apart on things like statistical reasoning if you switch domains (I think it is the Leda Cosmides evo psych stuff that goes into it but there might be a more famous experiment).

[deleted]
Post reply on HN