Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Simple tasks showing reasoning breakdown in state-of-the-art LLMs

arxiv.org

Post reply on HN
Print view View original

191–200 of 393 posts

Re: Simple tasks showing reasoning breakdown in state-of-the-art LLMs

#191

Post by rachofsunshine »

larkinnaire wrote: post #177

The idea that these word problems (and other LLM stumpers) are "easily solvable by humans" needs some empirical data behind it. Computer people like puzzles, and this kind of thing seems straightforward to them. I think the percentage of the general population who would get these puzzles right with the same time constraints LLMs are subjected to is much lower than the authors would expect, and that the LLMs are right…

Yeah, as someone with an education background I suspect GPT-4 is relatively close to the general public's performance on this problem. Many people would miss AIW, and almost all would miss AIW+. I'm about as good at this kind of thing as anyone and I'd need a minute with pencil and paper to handle AIW+; it's on par with the most difficult problems found on tests like the GRE.

I wonder if these models, trained on data from across the internet, are in some ethereal way capturing the cognitive approaches of the average person (and not picking the best approaches). If the average person does not think in these sorts of symbolic-manipulative terms, and therefore does not write in those terms, and you train a model on that writing...?

Re: Simple tasks showing reasoning breakdown in state-of-the-art LLMs

#192

Post by astrobe_ »

photon_lines wrote: post #44
nerdjon wrote: post #18

For anyone considering reading the paper and like me don't normally read papers like this, open the PDF and think you don't have time to read it due to its length. The main part of the paper is the first 10 pages and a fairly quick read. On to the topic here. This is an interesting example that they are using. It is fairly simplistic to understand as a human (even if we may be inclined to quickly jump to the wrong co…

There's actually a pretty simple solution to this that I thought about testing out and it involves asking the model to re-construct the problem using a logic language (like Prolog) and asking it to execute this type of program in order to come up with a solution rather than attempting simple chain-of-reason training / other methodologies of getting the model to 'reason' through some of these examples. People forget t…

> asking it to execute this type of program in order to come up with a solution

I may be showing my ignorance about this tech here, but I believe the LLM doesn't even try to solve a problem; they try to generate a discourse that could pass as a solution or answer to the problem; that's more or less what the abstract states if I understand it correctly. But in no way does it try to apply some sort of mechanical reasoning like inference engines do.

To me the solution to this is to associate LLM with mechanical computations, that is an inference engine or an equation solver, rather than recombining the millions of solutions for similar problems it has seen in its training set. I believe I remember reading about teams attempting this approach. I can imagine for instance that if the LLM is in some way able to ask questions and use the answer, maybe it could just generate a prompt for an equation solver and include the result in its answmer.

Re: Simple tasks showing reasoning breakdown in state-of-the-art LLMs

#193

Post by logicchains »

layer8 wrote: post #94
nerdjon wrote: post #18

For anyone considering reading the paper and like me don't normally read papers like this, open the PDF and think you don't have time to read it due to its length. The main part of the paper is the first 10 pages and a fairly quick read. On to the topic here. This is an interesting example that they are using. It is fairly simplistic to understand as a human (even if we may be inclined to quickly jump to the wrong co…

For reasoning of any complexity, some sort of internal monologue and iteration is needed. For type-2 thinking, we conceptualize possible solutions, arguments, and reasoning paths in our mind, and judge and evaluate them before deciding which one we think is adequate or correct. This can be emulated to some extent by guiding prompts that make that internal thought process external, but we really want it to be internal…

>For reasoning of any complexity, some sort of internal monologue and iteration is needed.

From a formal perspective you're entirely correct. Transformers with chain-of-thought are strictly more powerful than transformers without it, and can efficiently solve classes of problems that would otherwise require exponentially increasing model depth: https://arxiv.org/abs/2310.07923

Re: Simple tasks showing reasoning breakdown in state-of-the-art LLMs

#194

Post by staticman2 »

GeoAtreides wrote: post #154
glitchc wrote: post #132

Earlier quoted context omitted.

In this particular case, is there any reason why we simply can't take their word for it? This is not a case of where if I say "weak" or "strong", most people pick strong because no one wants to be weak, even if the context is unknown (nuclear force for example).

> is there any reason why we simply can't take their word for it? because if we give them a problem to solve in their head and just give us the answer, they will. By problem I mean planning a trip, a meal, how to pay the mortgage, etc. It's impossible to plan without an internal monologue. Even if some people claim theirs is 'in images'.

>> It's impossible to plan without an internal monologue

That's quite the claim.

Re: Simple tasks showing reasoning breakdown in state-of-the-art LLMs

#195

Post by glitchc »

slashdave wrote: post #178
glitchc wrote: post #131

Earlier quoted context omitted.

The problem is a good chunk of the global population is also not reasoning and thinking in any sense of the word. Logical reasoning is a higher order skill that often requires formal training. It's not a natural ability for human beings.

Seriously? You think individuals are incapable of reasoning without training first?

Yes, seriously.

Some examples: An individual without training cannot reliably separate cause from effect, or judge that both events A and B may have a common root cause. Similarly, people often confuse conditionals for causation. People often have difficulty reasoning about events based on statistical probabilities. Remember, the average person in North America is far more terrified of a terror attack than an accident or a heart attack, yet the latter two are much more likely to be their cause of death.

Re: Simple tasks showing reasoning breakdown in state-of-the-art LLMs

#196

Post by ElevenLathe »

GeoAtreides wrote: post #154
glitchc wrote: post #132

Earlier quoted context omitted.

In this particular case, is there any reason why we simply can't take their word for it? This is not a case of where if I say "weak" or "strong", most people pick strong because no one wants to be weak, even if the context is unknown (nuclear force for example).

> is there any reason why we simply can't take their word for it? because if we give them a problem to solve in their head and just give us the answer, they will. By problem I mean planning a trip, a meal, how to pay the mortgage, etc. It's impossible to plan without an internal monologue. Even if some people claim theirs is 'in images'.

> It's impossible to plan without an internal monologue.

How can science make this claim if it can't prove (or disprove) the existence of an internal monologue?

Re: Simple tasks showing reasoning breakdown in state-of-the-art LLMs

#197

Post by alfiopuglisi »

IlliOnato wrote: post #190

This is a cool one, but I know of other such "failures". For example, try to ask (better in Russian), how many letters "а" are there in Russian word "банан". It seems all models answer with "3". Playing with it reveals that apparently LLMs confuse Russian "банан" with English "banana" (same meaning). Trying to get LLMs to produce a correct answer results is some hilarity. I wonder if each "failure" of this kind deser…

No current LLM understands words, nor letters. They all have input and output tokens, that roughly correspond to syllabes and letter groupings. Any kind of task involving counting letters or words is outside their realistic capabilities.

LLMs are a tool, and like any other tool, they have strengths and weaknesses. Know your tools.

Re: Simple tasks showing reasoning breakdown in state-of-the-art LLMs

#198

Post by photon_lines »

GeoAtreides wrote: post #154
glitchc wrote: post #132

Earlier quoted context omitted.

In this particular case, is there any reason why we simply can't take their word for it? This is not a case of where if I say "weak" or "strong", most people pick strong because no one wants to be weak, even if the context is unknown (nuclear force for example).

> is there any reason why we simply can't take their word for it? because if we give them a problem to solve in their head and just give us the answer, they will. By problem I mean planning a trip, a meal, how to pay the mortgage, etc. It's impossible to plan without an internal monologue. Even if some people claim theirs is 'in images'.

'It's impossible to plan without an internal monologue.' - Sorry, but I disagree with this. I have no 'internal voice' or monologue - whenever I see a problem, my brain actually and fully models it using images. I believe 25% of the population doesn't have the internal monologue which you're referring to and this has been tested and confirmed. I highly recommend listening to this Lex Friedman podcast episode to get a full grasp on the complexities of modelling language and general modelling present in the human brain: https://www.youtube.com/watch?v=F3Jd9GI6XqE

Re: Simple tasks showing reasoning breakdown in state-of-the-art LLMs

#199

Post by Mmskynettio »

daveguy wrote: post #156
Mmskynettio wrote: post #136

Earlier quoted context omitted.

Its not an AI hype. A hype is defined as something which gets oversold: "promote or publicize (a product or idea) intensively, often exaggerating its benefits." Just yesterday I visited a google cloud summit and one person from bosch told the audiance how they are now able to work with less external agencies like texting, graphicsdesigner and photographers for their materials. It already saves money, has real impacts…

> We are also don't know what ChatGPT 5 will bring, because they say this will do more reasoning than before... This paper very clearly demonstrates these LLMs are not reasoning in a fundamental way. Token prediction and reasoning are two different tasks. They may be related, but they are not the same. "Just wait for GPT 5, it will be amazing!" is part of the hype. Please do not assume an LLM is correct in skill or k…

The problem is that we don't know how we do reasoning.

I calculate stuff by following a formular after i pattern detected a problem i already know.

Plenty of humans are not able to solve those math problems.

If the future of llm / ai becomes a LLM with multi modal and mixture of experts and that solves those reasoning problems, we still don't know if this is a different type of reasoning than what humans do.

Re: Simple tasks showing reasoning breakdown in state-of-the-art LLMs

#200

Post by solidasparagus »

indoordin0saur wrote: post #184
solidasparagus wrote: post #164

Earlier quoted context omitted.

> AI needs to see thousands or millions of images of a cat before they reliably can identify one. Not if they inherit from a previous generation of AI. But even if they did, a different training speed does not imply a different capability

My point is not that humans have a faster training speed but that humans must be doing something fundamentally different from LLMs. You could build Altman's $7 trillion dollar GPU cluster and use the majority of the world's energy to feed it and you'd still hit the same limitations if you're just running an LLM on it, even a very sophisticated LLM. This is Yann LeCun's position as well.

That, as I understand it, is not a valid chain of logic. Requiring fewer data points does not inherently indicate that the underlying mechanism (autogressive sequential generation, not the transformer which is just an architecture) is different.

Not to mention the secondary arguments like - no proof that human learns faster from fewer datapoints, that's just your assumption in the sibling comment. Humans inherit information. The equivalent - fine-tuning a foundation model - is very fast to learn novel objects.

Just because someone has a Turing award doesn't mean they know what they're talking about. They are just people, with strengths and weaknesses like everyone else. But often on the extreme end of strengths and weaknesses.

Post reply on HN