I really liked that paper, easy to understand and explain every terminology they used, even the variance computation is explained! I don't expect that much detail from every paper, but it's a bliss to read.
Simple tasks showing reasoning breakdown in state-of-the-art LLMs
31–40 of 393 posts
Re: Simple tasks showing reasoning breakdown in state-of-the-art LLMs
#32This is obvious... but some human knowledge outside the question is needed. (Alice is female, sister is female, brother is male, relation of sisters and brothers). In a school math "word problem" you don't need this kind of info.
Re: Simple tasks showing reasoning breakdown in state-of-the-art LLMs
#33For anyone considering reading the paper and like me don't normally read papers like this, open the PDF and think you don't have time to read it due to its length. The main part of the paper is the first 10 pages and a fairly quick read. On to the topic here. This is an interesting example that they are using. It is fairly simplistic to understand as a human (even if we may be inclined to quickly jump to the wrong co…
In many ways, this is very obvious and routine to people who use these systems with a critical understanding of how they work. It's dispiriting how we require a arxiv.org pdf in order to point out the absence of the emperor's clothing. There are few people who use LLMs who could not, right now, demonstrate this point to themselves if they so wish.
Such is the strength of corporate tech propaganda that a whole mass of people will instead insist that we have never worn clothes either.
Re: Simple tasks showing reasoning breakdown in state-of-the-art LLMs
#34I'm still waiting for people to combine LLMs with logic engines (prolog) or proof assistants. LLMs are afaik not designed to reason. But they can probably be used to construct formal statements that can be processed by logic engines.
Re: Simple tasks showing reasoning breakdown in state-of-the-art LLMs
#35Citation 40 is the longest list of authors I have ever seen. That is one way to help all your friends get tenure.
Re: Simple tasks showing reasoning breakdown in state-of-the-art LLMs
#36Question is: "Alice has 60 brothers and she also has 212 sisters. How many sisters does Alice’s brother have?" (nb: I have added numbers, it's phrased as X and N in the paper) I must confess, when I tried to answer the question I got it wrong...! (I feel silly). I only realised I got it wrong when I plugged it into GPT-4o and it came back with the correct answer: https://chatgpt.com/share/6eb5fa36-e0fd-4417-87d1-64ca…
> I must confess, when I tried to answer the question I got it wrong...! (I feel silly).
In programming there are two difficult problems - naming things, cache invalidation, and off-by-one error.
Re: Simple tasks showing reasoning breakdown in state-of-the-art LLMs
#37Question is: "Alice has 60 brothers and she also has 212 sisters. How many sisters does Alice’s brother have?" (nb: I have added numbers, it's phrased as X and N in the paper) I must confess, when I tried to answer the question I got it wrong...! (I feel silly). I only realised I got it wrong when I plugged it into GPT-4o and it came back with the correct answer: https://chatgpt.com/share/6eb5fa36-e0fd-4417-87d1-64ca…
Even "thinking out loud" we have seen these systems fall flat on their face and give very wrong answers.
Thinking out loud also only gets you so far, if the expectation is a certain type of response it can't always "think out loud". In reality that just proves it isn't really reasoning here and is more likely just self referencing.
That being said, I tried this personally allowing it to think out loud and it told me she has 212 sisters. Using your exact prompt.
Re: Simple tasks showing reasoning breakdown in state-of-the-art LLMs
#38Earlier quoted context omitted.
Great observation. It would be really interesting to repeat this research with less limiting prompts. I assume they made the restriction to make it easier to parse the answers, perhaps make it two phase, let them respond unlimited, then a follow up prompt ask to compress it to a single answer. I wonder how the results will vary.
To be quite honest, I assume they made the restriction so that the models would fail. This sort of paper is becoming a genre.
You test models where they fail in any field.
The orbit of Mercury to discover GR as an example.
As all models are wrong, but some are useful, finding where they fail is how you figure out if they are useful.
As the 'AGI is near' camp has won the hype game, it is important to ground expectations for practical exploitation of the technology.
Over promising unabashed optimism is partly what caused the previous AI winters.
As the formal proof methods of mathematics proved impractical, counterexamples and the scientific method is what CS has used for decades.
Re: Simple tasks showing reasoning breakdown in state-of-the-art LLMs
#39This is obvious... but some human knowledge outside the question is needed. (Alice is female, sister is female, brother is male, relation of sisters and brothers). In a school math "word problem" you don't need this kind of info.
LLMs perform well on “human knowledge” like you are describing. They don’t lack knowledge of simple relational facts like these.
Nonetheless it would be interesting to see the problem reformulated in purely mathematical terms, I suspect models would perform better.
That is the point though - models are showing an inability to generalize their capabilities from one domain (maths / abstract logic) into other domains (conversational reasoning).
Re: Simple tasks showing reasoning breakdown in state-of-the-art LLMs
#40For anyone considering reading the paper and like me don't normally read papers like this, open the PDF and think you don't have time to read it due to its length. The main part of the paper is the first 10 pages and a fairly quick read. On to the topic here. This is an interesting example that they are using. It is fairly simplistic to understand as a human (even if we may be inclined to quickly jump to the wrong co…
I don't think anybody who actually uses LLMs thinks they are general purpose AI.
Like you ask it to do one thing it's amazing, but then you try to modify or do something with extra steps, or just anything with any complexity to it and it falls over.