Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Simple tasks showing reasoning breakdown in state-of-the-art LLMs

arxiv.org

Post reply on HN
Print view View original

341–350 of 393 posts

Re: Simple tasks showing reasoning breakdown in state-of-the-art LLMs

#341

Post by tsunamifury »

TeMPOraL wrote: post #326
imtringued wrote: post #137

Earlier quoted context omitted.

No it is because supervised and self supervised learning happen to produce reasoning as a byproduct. For some reason people think that telling a model to recite a trillion tokens somehow will improve it beyond the recitation of those tokens. I mean, in theory you can select the training data so that it will learn what you want, but then again you are limited to what you taught it directly. The problem is that these m…

> The problem is that these models weren't trained to reason. Except they kind of were. Specifically, they were trained to predict next tokens based on text input, with the optimization function being, does the result make sense to a human? . That's embedded in the training data: it's not random strings, it's output of human reasoning, both basic and sophisticated. That's also what RLHF selects for later on. The mode…

I think people don’t recognize it’s currently doing single turn reasoning and demonstrating the building blocks of real time reasoning with continuous input.

If we tested humans on first thought questions and answers in 5 seconds or less on half the problems we did on LLMs — we might prove humans can’t reason as well

Re: Simple tasks showing reasoning breakdown in state-of-the-art LLMs

#342

Post by hombre_fatal »

TeMPOraL wrote: post #336
hombre_fatal wrote: post #333

Earlier quoted context omitted.

I'm responding to "why can't we just take their word for it?" That you and I can come up with different ways to describe our subjective experience in conversation doesn't mean that we have a different subjective experience. Especially not when relayed by a species that's frequently convinced it has a trending mental disorder from TikTok.

We can keep talking about it, and assuming we're both honest, we'll arrive at the answer to whether or not our subjective experiences differ. To fail at that would require us to have so little in common that we wouldn't be able to communicate at all . Which is obviously not the case, neither for us, nor for almost every possible pair of humans currently alive.

On the other hand, a deep one on one discussion isn't what's happening in casual debates online about to what degree each of us has an inner monologue. And because we don't have so little in common, I would be resistant to concluding that my subjective experience is so different than everyone else's. To claim that I'm different requires me to have an accurate model of what other people are experiencing, not just an accurate report of what I'm experiencing.

Look up examples of this on reddit and you'll find a lot of larping. I would take most of it with a grain of salt as you should with any story-telling you encounter on social media.

If we're so reliable, there wouldn't be fake mental illness epidemics on TikTok regarding experiences far more concrete than fuzzy notions like inner monologue.

Re: Simple tasks showing reasoning breakdown in state-of-the-art LLMs

#343

Post by elicksaur »

tsunamifury wrote: post #335
elicksaur wrote: post #330

Earlier quoted context omitted.

Maybe people have different experiences with the products than you. A simulated reasoning machine being possible does not mean that current LLMs are simulated thinking machines. Maybe you should try asking chatgpt for advice on how to understand other people’s perspectives: https://chatgpt.com/share/3d63c646-859b-4903-897e-9a0cb7e47b...

This is such a weirdly preachy and belligerent take. Obviously that was implied in my statement. Dude we aren’t all 4 year olds that need a self righteous lesson

Weird to accuse a response of being belligerent when your initial comment stated that people who disagreed with you are in denial.

What was implied by your statement? That you don’t understand other people’s perspectives?

Re: Simple tasks showing reasoning breakdown in state-of-the-art LLMs

#344

Post by TeMPOraL »

rprospero wrote: post #188
GeoAtreides wrote: post #154

Earlier quoted context omitted.

> is there any reason why we simply can't take their word for it? because if we give them a problem to solve in their head and just give us the answer, they will. By problem I mean planning a trip, a meal, how to pay the mortgage, etc. It's impossible to plan without an internal monologue. Even if some people claim theirs is 'in images'.

> It's impossible to plan without an internal monologue I once had a teacher claim that people who claimed to have aphantasia were lying, because those people have read books and it is impossible to read a book without picture the scene in your mind's eye. Are you citing the same source that she was?

I wish I had such a teacher, because I'd learn the term "aphantasia", instead of worrying all my youth that I'm doing reading wrong, as I could never picture anything I was reading in my mind (and as a result, I found scenery descriptions to be mind dumbingly boring).

Re: Simple tasks showing reasoning breakdown in state-of-the-art LLMs

#345

Post by dr_dshiv »

lupire wrote: post #234
dr_dshiv wrote: post #115

Earlier quoted context omitted.

Well, I believe LLMs are general purpose AI. And, for the record, so does Peter Norvig. He coauthored the most popular textbook on Artificial Intelligence, so this belief can be considered a somewhat mainstream perspective. His article, “Artificial General Intelligence Is Already Here“ [1] is really worth reading. https://www.noemamag.com/artificial-general-intelligence-is-...

I can't tell whether Norvig is making a subtle's point that artificial general stupidity is still an important form of artificial general intelligence, or he whether he somehow vastly overestimates AI model performance on non-memorization and summarization tasks.

You seem to have an axe to grind. You might spend more time trying to get quality out of the models. It is possible!

Re: Simple tasks showing reasoning breakdown in state-of-the-art LLMs

#346

Post by hughesjj »

twiceaday wrote: post #255
hifromwork wrote: post #205

Earlier quoted context omitted.

>I bet you can reason 1+3x=22 pretty easily without any words whatsoever I've tried to do it, but I can't. I had to do something like "ok, so we subtract one from both sides and then it's easy, 3*7=21". Maybe I could do 2+8 but I still think the word ten "aloud".

I was able to do it with no words. I 'saw' the steps as if on a piece of paper. I saw 3x=22-1=21, then x=21/3=7. But I have a degree in applied math. Perhaps not internally vocalizing is just being extremely familiar. It also happened very quickly, perhaps there was no time to vocalize anyways.

To be fair, math is a language in itself... with many dialects come to tcreditors.

At the end of the day though, thought requires communication, even if internal. Even physics is modelled as some sort of 'message passing' when we try to unravel what causality really is. Similar to how a processor has cycles, I think/know similar (but unsynced) happens as part of what we call 'thinking'.

Re: Simple tasks showing reasoning breakdown in state-of-the-art LLMs

#347

Post by nomel »

layer8 wrote: post #94
nerdjon wrote: post #18

For anyone considering reading the paper and like me don't normally read papers like this, open the PDF and think you don't have time to read it due to its length. The main part of the paper is the first 10 pages and a fairly quick read. On to the topic here. This is an interesting example that they are using. It is fairly simplistic to understand as a human (even if we may be inclined to quickly jump to the wrong co…

For reasoning of any complexity, some sort of internal monologue and iteration is needed. For type-2 thinking, we conceptualize possible solutions, arguments, and reasoning paths in our mind, and judge and evaluate them before deciding which one we think is adequate or correct. This can be emulated to some extent by guiding prompts that make that internal thought process external, but we really want it to be internal…

> but we really want it to be internal and automatic.

Why? We have specialized semi-isolated lobes of our brain. Some "external to the LLM weights" guiding system just needs to be transparent to us. It doesn't need to be internal to the LLM/weights.

> because what controls the internal thought process is itself not necessarily language-based

Most of the interesting AI things can't write a sentence (weather simulation, protein folding, inverse kinematics, driving cars, etc...). Language isn't a requirement, only some sort of meaningful mapping to the latent space of each is. I think one could claim that "meaningful latent space mapping" is the language of neural nets, including those in the brain.

Re: Simple tasks showing reasoning breakdown in state-of-the-art LLMs

#348

Post by hughesjj »

TeMPOraL wrote: post #309
ElevenLathe wrote: post #238

Earlier quoted context omitted.

Not only are they unproven, but are ultimately not provable at all. Some people will say yes, some people will say no. Probably we can take their word for it, but in the simplest case they could just lie (in either direction) and we would have no way to tell. In short, maybe these inner monologues exist and maybe they don't, but science can't comment on that. That said, it is clearly something we are interested in, b…

> Probably we can take their word for it, but in the simplest case they could just lie (in either direction) and we would have no way to tell. Individually, no, but in general, for people to consistently lie about this particular thing at scale would be extremely unusual, given that people rarely lie if there's no reason for it. Going by this baseline, you could assume upward of 50% of replies are honest (even if mis…

I've heard a theory where the inner monolog was emergent, and some of the first people to recognize thr 'voice in their heads' attributed it to god/angels/etc

There's conspiratorial lying and lying from ignorance, one is much less credulous.

Re: Simple tasks showing reasoning breakdown in state-of-the-art LLMs

#349

Post by danielmarkbruce »

IggleSniggle wrote: post #152
layer8 wrote: post #119

Earlier quoted context omitted.

It’s commonly conjectured that the emergence of human-level reasoning wouldn’t have been possible without the development of language. Personally, I’m able to suppress “word thoughts” in my head (for a short time), but then I lose almost all of my reasoning ability. I could imagine that reasoning is language-based even when it’s not conscious for some people. An internal process being there, and being conscious of it…

Maybe, but symbolic thought can get pretty far away from what we generally call "language." I bet you can reason 1+3x=22 pretty easily without any words whatsoever, or the sound of one ascending octave after another, or the approximate G-force induced on your body if you take the next turn without applying the brakes. All of these forms of reasoning are true and useful calculations: when we talk about "intuition" wha…

Most people can't do 1 + 3x = 22 without any words or symbols. People who can don't realize that most people can't. I'd argue one isn't using logic when they do that, it's just very good pattern matching.

Re: Simple tasks showing reasoning breakdown in state-of-the-art LLMs

#350

Post by ninetyninenine »

nerdjon wrote: post #18

For anyone considering reading the paper and like me don't normally read papers like this, open the PDF and think you don't have time to read it due to its length. The main part of the paper is the first 10 pages and a fairly quick read. On to the topic here. This is an interesting example that they are using. It is fairly simplistic to understand as a human (even if we may be inclined to quickly jump to the wrong co…

> They are not reasoning or thinking in any sense of the word and yet a lot of people are already considering them general purpose AI.

You yourself did not use reasoning to arrive at this conclusion. It's quite obvious. I'm not trying to belittle you here. But LLMs are black boxes, we do not actually know what they are doing at a high enough resolution where we can call it "not reasoning" or "reasoning".

We can only characterize these AI's as a best fit curve between datapoints which is a way to high level view point to come to any conclusion about "reasoning"

This paper presents evidence of failed reasoning, but how does that prove anything when LLMs exhibit many instances of successful reasoning on complex topics they were not trained on?

You are biased and honing into information that supports a biased conclusion. LLMs are an AI we do not understand at a low level. Hence we talk about the attributes of these AIs in the same way we talk about humans, "Oh the LLM hallucinates", "it tries to justify it's answer..." etc. etc.

You characterize the Danger of these AI's as the result of Human stupidity. The danger according to you is solely from a human mistakenly believing that the AI is anything other than a stochastic parrot.

This is a belief arrived at in the same spirit as your claim. You did not use reasoning to arrive here.

The only logical way to characterize what is going on is that we do not know. It could very well be that these AI's are in fact reasoning. And that in itself presents a different kind of danger. A danger that may be more clear in the far future.

The irony is that your conclusion lacking correct reasoning is similarly parallel to the LLM's lack of reasoning. LLMs are more alike to us than you would like to believe.

Can you give me the step by step signal path ways of an LLM processing a query to prove that it does not reason? Or do you have to use black box anecdotal evidence to prove your point? For any "evidence" where an LLM failed to reason, there is another counter example showing where the LLM succeeded. Contradictory evidence can only lead to a vague conclusion.

Post reply on HN