Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Simple tasks showing reasoning breakdown in state-of-the-art LLMs

arxiv.org

Post reply on HN
Print view View original

151–160 of 393 posts

Re: Simple tasks showing reasoning breakdown in state-of-the-art LLMs

#151

Post by [deleted] »

solidasparagus wrote: post #128
ericmcer wrote: post #118

Earlier quoted context omitted.

If you really think about what an LLM is you would think there is no way that leads to general purpose AI. At the same time though they are already doing way more than we thought they could. Maybe people were surprised by what OpenAI achieved so now they are all just praying that with enough compute and the right model AGI will emerge.

> If you really think about what an LLM is you would think there is no way that leads to general purpose AI It is an autoregressive sequence predictor/generator. Explain to me how humans are fundamentally different

[deleted]

Re: Simple tasks showing reasoning breakdown in state-of-the-art LLMs

#152

Post by IggleSniggle »

layer8 wrote: post #119
sollewitt wrote: post #105

Earlier quoted context omitted.

Given many people don’t have an inner monologue and function just fine, it’s more likely inner monologue is a product of the reasoning process and not it’s mechanism.

It’s commonly conjectured that the emergence of human-level reasoning wouldn’t have been possible without the development of language. Personally, I’m able to suppress “word thoughts” in my head (for a short time), but then I lose almost all of my reasoning ability. I could imagine that reasoning is language-based even when it’s not conscious for some people. An internal process being there, and being conscious of it…

Maybe, but symbolic thought can get pretty far away from what we generally call "language." I bet you can reason 1+3x=22 pretty easily without any words whatsoever, or the sound of one ascending octave after another, or the approximate G-force induced on your body if you take the next turn without applying the brakes.

All of these forms of reasoning are true and useful calculations: when we talk about "intuition" what we usually mean is that we have a lot of experience and internal reasoning about a subject, but we struggle to translate it to and from the "language" part of our brain. Nonetheless, any social dancer will tell you that a dialog is possible just by receiving and inducing g-forces alone. You can reason this way about abstract concepts like orbits without ever touching a single word or concrete symbol.

Edit: the key aspect of reasoning, imho, is the ability to make predictions and introspect them against a database of other predictions, using an adversarial heuristic to weight the most plausibly useful results. Perhaps our pattern matching AIs of today just lack sufficient "experience" to do what we call reasoning.

Re: Simple tasks showing reasoning breakdown in state-of-the-art LLMs

#153

Post by kalkin »

nerdjon wrote: post #41
mjburgess wrote: post #33

Earlier quoted context omitted.

In many ways, this is very obvious and routine to people who use these systems with a critical understanding of how they work. It's dispiriting how we require a arxiv.org pdf in order to point out the absence of the emperor's clothing. There are few people who use LLMs who could not, right now, demonstrate this point to themselves if they so wish. Such is the strength of corporate tech propaganda that a whole mass of…

> In many ways, this is very obvious and routine to people who use these systems with a critical understanding of how they work. The last part of that is the problem and why a paper like this is critical. These systems are being pushed onto people who don't understand how they work. CEO's and other business leaders are being pushed to use AI. Average users are being shown it in Google search results. Etc etc. People…

> People are being told it can do far more than it really is.

Meanwhile these HN comments are split between:

* Lots of people confirming what the paper itself notes (but doesn't highlight), that the most advanced models actually can solve this problem at least a significant portion of the time. (A proportion which one can pretty easily project is only likely to increase with future models.)

* Lots of people saying "this confirms LLMs can't do reasoning".

Questions I'd ask you to consider:

* Is "LLMs can't do reasoning" actually more accurate than the typical hype?

* Is a "critical understanding of how [LLMs] work" that would predict they simply cannot solve this problem actually a good understanding?

Re: Simple tasks showing reasoning breakdown in state-of-the-art LLMs

#154

Post by GeoAtreides »

glitchc wrote: post #132
ElevenLathe wrote: post #129

Earlier quoted context omitted.

The existence of an "inner monologue" isn't really a falsifiable claim. Some people claim to have one while other people claim not to, but we can't test the truth of these claims.

In this particular case, is there any reason why we simply can't take their word for it? This is not a case of where if I say "weak" or "strong", most people pick strong because no one wants to be weak, even if the context is unknown (nuclear force for example).

> is there any reason why we simply can't take their word for it?

because if we give them a problem to solve in their head and just give us the answer, they will. By problem I mean planning a trip, a meal, how to pay the mortgage, etc. It's impossible to plan without an internal monologue. Even if some people claim theirs is 'in images'.

Re: Simple tasks showing reasoning breakdown in state-of-the-art LLMs

#155

Post by pbhjpbhj »

ip26 wrote: post #113
nerdjon wrote: post #54

Earlier quoted context omitted.

I honestly thought about this recently when I was trying to see the limits of Claude Opus. Some of the problems I gave it, what if instead of telling it to solve the problem I asked it to write the script and then give me the command and inputs needed to properly run it to get the answer I needed. That way instead of relying on the LLM to do properly analysis of the numbers it just needs to understand enough to write…

I don’t understand why LLM’s aren’t already set up to do what you describe automatically behind the curtain. Extract a math equation from text (LLMs are good at translating between languages right?) and immediately evaluate it on the host CPU. LLM is the equivalent of recalling your times tables. Computer arithmetic is the equivalent of re-computing your times tables.

ChatGPT does do this sort of process for arithmetic now; it converts wordbased problems to mathematical notation and then solves.

Re: Simple tasks showing reasoning breakdown in state-of-the-art LLMs

#156

Post by daveguy »

Mmskynettio wrote: post #136
nerdjon wrote: post #18

For anyone considering reading the paper and like me don't normally read papers like this, open the PDF and think you don't have time to read it due to its length. The main part of the paper is the first 10 pages and a fairly quick read. On to the topic here. This is an interesting example that they are using. It is fairly simplistic to understand as a human (even if we may be inclined to quickly jump to the wrong co…

Its not an AI hype. A hype is defined as something which gets oversold: "promote or publicize (a product or idea) intensively, often exaggerating its benefits." Just yesterday I visited a google cloud summit and one person from bosch told the audiance how they are now able to work with less external agencies like texting, graphicsdesigner and photographers for their materials. It already saves money, has real impacts…

> We are also don't know what ChatGPT 5 will bring, because they say this will do more reasoning than before...

This paper very clearly demonstrates these LLMs are not reasoning in a fundamental way. Token prediction and reasoning are two different tasks. They may be related, but they are not the same. "Just wait for GPT 5, it will be amazing!" is part of the hype.

Please do not assume an LLM is correct in skill or knowledge unless you already know the answer or can verify by other means.

Re: Simple tasks showing reasoning breakdown in state-of-the-art LLMs

#157

Post by indoordin0saur »

solidasparagus wrote: post #128
ericmcer wrote: post #118

Earlier quoted context omitted.

If you really think about what an LLM is you would think there is no way that leads to general purpose AI. At the same time though they are already doing way more than we thought they could. Maybe people were surprised by what OpenAI achieved so now they are all just praying that with enough compute and the right model AGI will emerge.

> If you really think about what an LLM is you would think there is no way that leads to general purpose AI It is an autoregressive sequence predictor/generator. Explain to me how humans are fundamentally different

AI needs to see thousands or millions of images of a cat before they reliably can identify one. The fact that a child needs to only see one example of a cat to know what a cat is from then on seems to point to humans having something very different.

Re: Simple tasks showing reasoning breakdown in state-of-the-art LLMs

#158

Post by Mmskynettio »

Lich wrote: post #146
Mmskynettio wrote: post #136

Earlier quoted context omitted.

Its not an AI hype. A hype is defined as something which gets oversold: "promote or publicize (a product or idea) intensively, often exaggerating its benefits." Just yesterday I visited a google cloud summit and one person from bosch told the audiance how they are now able to work with less external agencies like texting, graphicsdesigner and photographers for their materials. It already saves money, has real impacts…

> AI doesn't need to be ready tomorrow, but its capabilities are already really good. A majority don’t deny that it’s good. The problem is that so many think it is actually reasoning, believing the answers can be trusted.

Does it need to do reasoning perfectly?

If it created meta concepts from billion words on the internet and has meta models which are correct and are more and better than an avg human, isn't it actually good in reasoning?

Its a very narrow thing to say 'is that so many think its actually reasoning' to say AI is just hype or everything we are doing is a waste etc.

There are human benchmarks they are winning at. The critic could be more that we don't have enough benchmarks.

Re: Simple tasks showing reasoning breakdown in state-of-the-art LLMs

#159

Post by solidasparagus »

throwanem wrote: post #147
solidasparagus wrote: post #128

Earlier quoted context omitted.

> If you really think about what an LLM is you would think there is no way that leads to general purpose AI It is an autoregressive sequence predictor/generator. Explain to me how humans are fundamentally different

"Prove me wrong?" That's not how this works. Your implicit claim here is that human cognition and LLM functioning are fundamentally similar. That claim requires substantiation.

It is how it works if you are replying to someone who claims "If you really think about what an LLM is you would think there is no way that leads to general purpose AI". The counter example is human beings are considered general purpose intelligence and we are complex, but fundamentally predictable systems (not by us today), with (as far as we can tell) deterministic outputs based on the state of the universe (including physical being like brain and chemicals).

Re: Simple tasks showing reasoning breakdown in state-of-the-art LLMs

#160

Post by photon_lines »

throwanem wrote: post #147
solidasparagus wrote: post #128

Earlier quoted context omitted.

> If you really think about what an LLM is you would think there is no way that leads to general purpose AI It is an autoregressive sequence predictor/generator. Explain to me how humans are fundamentally different

"Prove me wrong?" That's not how this works. Your implicit claim here is that human cognition and LLM functioning are fundamentally similar. That claim requires substantiation.

I actually did a full write-up on this here fyi: https://photonlines.substack.com/p/intuitive-and-visual-guid.... You can skip most of this and scroll down to the end-section called 'The Mental Model for Understanding LLMs' where I try to map how transformers are able to mimic human thinking. I think that comparing them to auto-associative / auto-regressive networks is actually a really good analogy FYI and I do believe a lot of human learning is based on this.
Post reply on HN