Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Simple tasks showing reasoning breakdown in state-of-the-art LLMs

arxiv.org

Post reply on HN
Print view View original

11–20 of 393 posts

Re: Simple tasks showing reasoning breakdown in state-of-the-art LLMs

#11

Post by eranation »

Closi wrote: post #3

Question is: "Alice has 60 brothers and she also has 212 sisters. How many sisters does Alice’s brother have?" (nb: I have added numbers, it's phrased as X and N in the paper) I must confess, when I tried to answer the question I got it wrong...! (I feel silly). I only realised I got it wrong when I plugged it into GPT-4o and it came back with the correct answer: https://chatgpt.com/share/6eb5fa36-e0fd-4417-87d1-64ca…

Great observation. It would be really interesting to repeat this research with less limiting prompts. I assume they made the restriction to make it easier to parse the answers, perhaps make it two phase, let them respond unlimited, then a follow up prompt ask to compress it to a single answer. I wonder how the results will vary.

Re: Simple tasks showing reasoning breakdown in state-of-the-art LLMs

#12

Post by [deleted] »

bamboozled wrote: post #6
Closi wrote: post #3

Question is: "Alice has 60 brothers and she also has 212 sisters. How many sisters does Alice’s brother have?" (nb: I have added numbers, it's phrased as X and N in the paper) I must confess, when I tried to answer the question I got it wrong...! (I feel silly). I only realised I got it wrong when I plugged it into GPT-4o and it came back with the correct answer: https://chatgpt.com/share/6eb5fa36-e0fd-4417-87d1-64ca…

I got it in 5 seconds, am I the singularity ?

[deleted]

Re: Simple tasks showing reasoning breakdown in state-of-the-art LLMs

#13

Post by gwbas1c »

naikus wrote: post #8

I'm using chatGPT 4o and it gave me the following :) ME Alice has 40 brothers and she also has 50 sisters. How many sisters does Alice's brother have? ChatGPT To determine how many sisters Alice's brother has, we need to use the information provided: Alice has 40 brothers and 50 sisters. Since Alice's brother is one of the 40 brothers, he shares the same family structure as Alice. This means he has the same number of…

Alice has 50 sisters.

Alice's brother has Alice as a sister (1) + Alice's sisters (50).

Therefore Alice's brother has 51 sisters.

Re: Simple tasks showing reasoning breakdown in state-of-the-art LLMs

#14

Post by FeepingCreature »

GPerson wrote: post #10
Closi wrote: post #3

Question is: "Alice has 60 brothers and she also has 212 sisters. How many sisters does Alice’s brother have?" (nb: I have added numbers, it's phrased as X and N in the paper) I must confess, when I tried to answer the question I got it wrong...! (I feel silly). I only realised I got it wrong when I plugged it into GPT-4o and it came back with the correct answer: https://chatgpt.com/share/6eb5fa36-e0fd-4417-87d1-64ca…

There must be a name for the new phenomenon, of which your post is an example, of: 1. Someone expresses that an LLM cannot do some trivial task. 2. Another person declares that they cannot do the task, thereby defending the legitimacy of the LLM. As a side note, I cannot believe that the average person who can navigate to a chatgpt prompter would fail to correctly answer this question given sufficient motivation to d…

Well, why does it need a name? It just seems like a "correct argument". If somebody is claiming that AI is "not all that" because it can make stupid mistakes, surely it's relevant that humans also make stupid mistakes, so AI making them does not necessarily pose a hindrance to it becoming a human-tier reasoner.

Re: Simple tasks showing reasoning breakdown in state-of-the-art LLMs

#15

Post by FeepingCreature »

eranation wrote: post #11
Closi wrote: post #3

Question is: "Alice has 60 brothers and she also has 212 sisters. How many sisters does Alice’s brother have?" (nb: I have added numbers, it's phrased as X and N in the paper) I must confess, when I tried to answer the question I got it wrong...! (I feel silly). I only realised I got it wrong when I plugged it into GPT-4o and it came back with the correct answer: https://chatgpt.com/share/6eb5fa36-e0fd-4417-87d1-64ca…

Great observation. It would be really interesting to repeat this research with less limiting prompts. I assume they made the restriction to make it easier to parse the answers, perhaps make it two phase, let them respond unlimited, then a follow up prompt ask to compress it to a single answer. I wonder how the results will vary.

To be quite honest, I assume they made the restriction so that the models would fail.

This sort of paper is becoming a genre.

Re: Simple tasks showing reasoning breakdown in state-of-the-art LLMs

#16

Post by oidar »

Closi wrote: post #3

Question is: "Alice has 60 brothers and she also has 212 sisters. How many sisters does Alice’s brother have?" (nb: I have added numbers, it's phrased as X and N in the paper) I must confess, when I tried to answer the question I got it wrong...! (I feel silly). I only realised I got it wrong when I plugged it into GPT-4o and it came back with the correct answer: https://chatgpt.com/share/6eb5fa36-e0fd-4417-87d1-64ca…

Of course it's going to give an incorrect answer with that prompt. If the instruction fine tuning is neutered like this prompt, it's going to roll over to the foundation model and offer a completion - probably more influenced by the seed than the prompting text. Bad study.

Edit - I just skimmed the paper - they do use other more appropriate prompt types for reasoning. My initial response was based on the assumption that all prompts used that script prompt quoted in the parent. I retract my "bad paper" comment.

Re: Simple tasks showing reasoning breakdown in state-of-the-art LLMs

#17

Post by [deleted] »

Closi wrote: post #3

Question is: "Alice has 60 brothers and she also has 212 sisters. How many sisters does Alice’s brother have?" (nb: I have added numbers, it's phrased as X and N in the paper) I must confess, when I tried to answer the question I got it wrong...! (I feel silly). I only realised I got it wrong when I plugged it into GPT-4o and it came back with the correct answer: https://chatgpt.com/share/6eb5fa36-e0fd-4417-87d1-64ca…

[deleted]

Re: Simple tasks showing reasoning breakdown in state-of-the-art LLMs

#18

Post by nerdjon »

For anyone considering reading the paper and like me don't normally read papers like this, open the PDF and think you don't have time to read it due to its length. The main part of the paper is the first 10 pages and a fairly quick read.

On to the topic here. This is an interesting example that they are using. It is fairly simplistic to understand as a human (even if we may be inclined to quickly jump to the wrong conclusion without thinking for a few seconds).

The thing that really bothers me is that I just don't know is realistically we can fix this given the current state of what these tools actually are. They are not reasoning or thinking in any sense of the word and yet a lot of people are already considering them general purpose AI. It doesn't help that in many situations it can fake it enough that it appears to be reasoning, but it's not.

What is the chance that this paper actually has any impact on the AI rollout and overhype or will just be buried and never talked about again until the next time we see how dangerous these tools are like with Google's search rollout.

Re: Simple tasks showing reasoning breakdown in state-of-the-art LLMs

#19

Post by breck »

It's an interesting paper, but a big flaw is that this:

> Alice has N brothers and she also has M sisters. How many sisters does Alice’s brother have?

...is not actually a simple task.

This can be quantified.

"1 + 1" is a simple task. It has a pretty small Total Complexity[1].

But to represent their task starting only with 1 and 0, you have to build a program of many, many lines. Orders of magnitude bigger than "1 + 1". Concepts like "has", "brother", "sister", "person", et cetera, have to be defined and built up.

[1] Counting Complexity (2017) https://github.com/breck7/breckyunits.com/blob/main/research...

Re: Simple tasks showing reasoning breakdown in state-of-the-art LLMs

#20

Post by sdiepend »

Closi wrote: post #3

Question is: "Alice has 60 brothers and she also has 212 sisters. How many sisters does Alice’s brother have?" (nb: I have added numbers, it's phrased as X and N in the paper) I must confess, when I tried to answer the question I got it wrong...! (I feel silly). I only realised I got it wrong when I plugged it into GPT-4o and it came back with the correct answer: https://chatgpt.com/share/6eb5fa36-e0fd-4417-87d1-64ca…

The right answer depends on how Alice identifies I guess? :)
Post reply on HN