Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Simple tasks showing reasoning breakdown in state-of-the-art LLMs

arxiv.org

Post reply on HN
Print view View original

281–290 of 393 posts

Re: Simple tasks showing reasoning breakdown in state-of-the-art LLMs

#281

Post by Jackson__ »

> Taking inspiration from those problems and aiming for even simpler settings, we arrived at a very

> simple problem template that can be easily solved using common sense reasoning but is not entirely

> straightforward, of the following form: "Alice has N brothers and she also has M sisters. How

> many sisters does Alice’s brother have?".

Cute paper, but claiming they invented it, when it's an extremely basic variant of a common[0] riddle asked of LLMs, is kinda weird.

[0] https://news.ycombinator.com/item?id=39786666#39794890 (Me, referencing the riddle, 74 days ago, having known it for a good while back then.)

Re: Simple tasks showing reasoning breakdown in state-of-the-art LLMs

#282

Post by program_whiz »

sys32768 wrote: post #57

As a non-coder I can get away with asking this: Why is it so hard to simulate reason? Logic and reason are based on rules. Then you add values to steer the conclusions based on the available data. Why not have separate systems for values and logic and memory working together as an AI brain to generate truly reasoned responses? You could even have adversarial parts that duke it out (left-wing vs right-wing, Jefferson…

you are describing GOFAI (good old fashioned AI) which was using things like logical production rules. The issue is that the ability to model the complexity of things like human languages and continuous/real-valued systems makes it infeasible to use at scale. For something like the alice brothers/sisters example, that is a canonical example where GOFAI and/or logical reasoning with languages like prolog excel. A common example in prolog is:

philosophers => human human => mortal plato => philosopher plato mortal? Yes

But encoding enough semantic information to create compelling AI with this type of system is difficult. Some have tried to enter thousands/millions of rules and still the system isn't convincing.

The main breakthrough that has enabled LLMs is an encoding of words that relies on their frequency in being near other words in the english language (using all the text available on the internet). Therefore words like "philosopher" and "plato" become associated in a high-dimensional space (so instead of "plato" you have a "token" with thousands of numbers associated with it).

You can then perform numeric operations on these numbers to come to conclusions. For example, we would expect something like a "human name" to emerge in this embedding space where we could determine if something "is used like a name" in various contexts by applying some non-linear transformations of the word vector / token.

LLMs have simply make a sequence of these transforms, while using prior words it generates as additional input (which allows it to construct sentences). So it is quite different from traditional reasoning. It is better at "fuzzy reasoning" but also worse in situations that require precise results (in fact, at each step it chooses one of a few best possible words based on its stats at random, the variation in this is called 'temperature').

Re: Simple tasks showing reasoning breakdown in state-of-the-art LLMs

#283

Post by skydhash »

lupire wrote: post #244
michael-ax wrote: post #86

Earlier quoted context omitted.

Alice has N Brothers, and she has M sisters. How many sisters do Alice’s brothers have? I have not gotten the correct answer to the question as phrased above in one go from Gpt4o yet! (and today was not the first day i tried.) Phrase it as shown above and you'll likely need 5 or more interactions to get it to generate the correct output. With Gemini i could not get it below 8 without feeling like i was cheating. fwiw…

Chat GPT 4o. I was being a bit generous with background information, but still tests ability to interpret: ------ Me: Background facts: Alice is a female human. All sisters are female, and all brothers are male. No one is their own brother or sister. Alice has N brothers, and Alice has M sisters. Now, a few questions based on these facts: How many sisters do Alice’s brothers have? Do Alice's brothers have more sister…

You: (In Prolog-like pseudocode)

  female(alice)
  sister_of(X, Y) :- !eq(X, Y), female(X), family_of(X, Y)
  brother_of(X, Y) :- !eq(X, Y), male(X), family_of(X, Y)
  [M] :- sister_of(M, alice)
  [N] :- brother_of(N, alice)
  
  [A] :- any([N], B), sister_of(A, B)
  count([A])?
  
  gt([A], [M])?
  eq([A], [M])?
  lt([A], [M])?
---

I don't know the exact encoding and decoding mechanism that ChatGPT 4o has, but I'm pretty sure all the basic facts and rules is already encoded by the models. And you conveniently added the rules that encode the puzzle itself.

Re: Simple tasks showing reasoning breakdown in state-of-the-art LLMs

#284

Post by gweinberg »

GPerson wrote: post #10
Closi wrote: post #3

Question is: "Alice has 60 brothers and she also has 212 sisters. How many sisters does Alice’s brother have?" (nb: I have added numbers, it's phrased as X and N in the paper) I must confess, when I tried to answer the question I got it wrong...! (I feel silly). I only realised I got it wrong when I plugged it into GPT-4o and it came back with the correct answer: https://chatgpt.com/share/6eb5fa36-e0fd-4417-87d1-64ca…

There must be a name for the new phenomenon, of which your post is an example, of: 1. Someone expresses that an LLM cannot do some trivial task. 2. Another person declares that they cannot do the task, thereby defending the legitimacy of the LLM. As a side note, I cannot believe that the average person who can navigate to a chatgpt prompter would fail to correctly answer this question given sufficient motivation to d…

Its not just that the LLM fails, its the kind of failures it makes that illustrate how far what the LLM does is from human thinking. Like the problem someone posted where you have a boat and a goat and you want to cross the river, and the LLM's solution is cross the river with the goat, come back with an empty boat, and then cross again with an empty boat. A human might miss the insight that Alice's brother has one more sister than Alice, namely Alice herself. A human would never think the number of sisters her brother has depends on the number of brothers she has. Not if it has any clue at all what "sister" and "brother" mean.

Re: Simple tasks showing reasoning breakdown in state-of-the-art LLMs

#285

Post by mcguire »

photon_lines wrote: post #44
nerdjon wrote: post #18

For anyone considering reading the paper and like me don't normally read papers like this, open the PDF and think you don't have time to read it due to its length. The main part of the paper is the first 10 pages and a fairly quick read. On to the topic here. This is an interesting example that they are using. It is fairly simplistic to understand as a human (even if we may be inclined to quickly jump to the wrong co…

There's actually a pretty simple solution to this that I thought about testing out and it involves asking the model to re-construct the problem using a logic language (like Prolog) and asking it to execute this type of program in order to come up with a solution rather than attempting simple chain-of-reason training / other methodologies of getting the model to 'reason' through some of these examples. People forget t…

I just tried that with ChatGPT 3.5 (4o stopped responding after I asked the initial question and it produced the wrong answer). Here's the Prolog it generated:

    % Define the number of brothers and sisters
    brothers(4).
    sisters(1).

    % Predicate to calculate the number of sisters a brother has
    brother_sisters(NumberOfSisters) :-
        sisters(NumberOfSisters).

    % Query to find out how many sisters a brother has
    ?- brother_sisters(Sisters).

Re: Simple tasks showing reasoning breakdown in state-of-the-art LLMs

#286

Post by FeepingCreature »

qsi wrote: post #79
FeepingCreature wrote: post #15

Earlier quoted context omitted.

To be quite honest, I assume they made the restriction so that the models would fail. This sort of paper is becoming a genre.

They used three different kinds of prompts with varying levels of restrictions, as described in the paper. To be quite honest, I assume you made your comment so that you could dismiss the paper without reading it.

That's a fair cop, I didn't read it.

The thing is that "LLM reasoning breaks down" simply did not surprise me enough that I thought it was worth clicking. Making LLMs fail is not hard. They're interesting for the ways that they work, not the (many, many) ways that they don't.

edit: I've had a look and I don't think any of their prompts are very good. They're certainly not how I'd write them if I wanted a current model to actually solve the problem.

The way to make me take a paper like this seriously would be if you set it up as an adversarial collaboration with a competent prompter, and that person agreed they couldn't make a generic prompt that solved the problem. "We tried three times and none worked" is not news, or at any rate not news about LLMs.

Re: Simple tasks showing reasoning breakdown in state-of-the-art LLMs

#287

Post by hombre_fatal »

glitchc wrote: post #132
ElevenLathe wrote: post #129

Earlier quoted context omitted.

The existence of an "inner monologue" isn't really a falsifiable claim. Some people claim to have one while other people claim not to, but we can't test the truth of these claims.

In this particular case, is there any reason why we simply can't take their word for it? This is not a case of where if I say "weak" or "strong", most people pick strong because no one wants to be weak, even if the context is unknown (nuclear force for example).

Because we can't be sure whether two people interpret what "inner monologue" means and whether they think it describes a phenomenon that actually isn't different between them and other people.

For example, I can think of interpretations of "I picture objects that I'm thinking about" that range from me not experiencing the phenomenon to me indeed experiencing the phenomenon.

To say that you're not experiencing something that other people are experiencing in their head is a solipsistic notion where you hypothesize an experience that you imagine others are having and then discard it for being different than yours.

Re: Simple tasks showing reasoning breakdown in state-of-the-art LLMs

#288

Post by throwanem »

solidasparagus wrote: post #159
throwanem wrote: post #147

Earlier quoted context omitted.

"Prove me wrong?" That's not how this works. Your implicit claim here is that human cognition and LLM functioning are fundamentally similar. That claim requires substantiation.

It is how it works if you are replying to someone who claims "If you really think about what an LLM is you would think there is no way that leads to general purpose AI". The counter example is human beings are considered general purpose intelligence and we are complex, but fundamentally predictable systems (not by us today), with (as far as we can tell) deterministic outputs based on the state of the universe (includ…

Responding to an unsubstantiated claim with an unsubstantiated claim just adds another unsubstantiated claim.

So far as I know, whether the universe behaves deterministically remains an unsolved question. Given that, your statement here would already be one of belief rather than fact, even before we get to the parentheticals. There is information here, but not about whether LLMs can develop into AGI.

Re: Simple tasks showing reasoning breakdown in state-of-the-art LLMs

#289

Post by mcguire »

photon_lines wrote: post #80
nerdjon wrote: post #54

Earlier quoted context omitted.

I honestly thought about this recently when I was trying to see the limits of Claude Opus. Some of the problems I gave it, what if instead of telling it to solve the problem I asked it to write the script and then give me the command and inputs needed to properly run it to get the answer I needed. That way instead of relying on the LLM to do properly analysis of the numbers it just needs to understand enough to write…

I'm not sure what you mean by 'it will not scale well.' When we humans learn that we make a mistake - we make a note and we hold the correct answer in memory - the next time we're prompted with a similar prompt, we can use our old memories to come up with the correct solution. I just did a simple test for this same exact problem using ChatGPT 3.5: 'Can you reformulate the following problem using Prolog? When you exec…

"My prompt after ChatGPT gave me an answer of 5: 'Yes but you aren't including Alice in your logic. Shouldn't the correct answer be 6 (since Alice is also a sister)?'"

Useful, if you know what the answer is. What happens if you don't give it the correct answer?

Re: Simple tasks showing reasoning breakdown in state-of-the-art LLMs

#290

Post by glitchc »

slashdave wrote: post #222
glitchc wrote: post #195

Earlier quoted context omitted.

Yes, seriously. Some examples: An individual without training cannot reliably separate cause from effect, or judge that both events A and B may have a common root cause. Similarly, people often confuse conditionals for causation. People often have difficulty reasoning about events based on statistical probabilities. Remember, the average person in North America is far more terrified of a terror attack than an acciden…

You mean without training, people cannot frame answers in the terms you've learned from training. Well, why are you surprised? If you think reasoning is limited to the frameworks you learned from a book, you live in a small world.

In an effort to be pejorative, you have reduced the process of learning to a basic regurgitation of facts. Rather, learning about reasoning is more about developing a framework that allows one to construct claims based on available information and then check the veracity of those claims while avoiding all, or at least typical, fallacies.

Critical thinking requires a certain amount of rigor, which formal education is well-suited to impart. It can be self-taught with a hefty dose of discipline, but it cannot be intuited.

Post reply on HN