Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Do LLMs pass the mirror test?

blog.pascalschuster.de

Post reply on HN
Print view View original

51–60 of 72 posts

Re: Do LLMs pass the mirror test?

#51

Post by [deleted] »

aspenmartin wrote: post #49
gmueckl wrote: post #44

Earlier quoted context omitted.

What else is there to say? LLMs can at most regurgitate approximations of human reasoning steps in the limited forms in which they may be expressed in the training data or interpolations thereof. That's the core essence of what they are. There is no proper reasoning to be found.

"at most" is wrong. RL with verifiable rewards takes you beyond quality and skills represented in training data, I'm not aware of meaningful fundamental limits here if you scale compute enough even though right now it's highly sample inefficient. Since you refuse to actually define what you consider to be reasoning let me at least put one out there: a system exhibits reasoning when an answer depends on nontrivial int…

[deleted]

Re: Do LLMs pass the mirror test?

#52

Post by gmueckl »

aspenmartin wrote: post #50
gmueckl wrote: post #47

Earlier quoted context omitted.

Reasoning includes things like proper use of logic. LLMs have been repeatedly shown to fail horribly at this. They consistently fail at drawing basic logical conclusions because they cannot build a sufficiently abstract model of certain problems that allows them to grasp their true nature. In other words, the whole class of questions of the kind of "how many r's in strawberry" or "do I take the car to the car wash?"…

> Reasoning includes things like proper use of logic. LLMs have been repeatedly shown to fail horribly at this. That models cannot do ALL logic problems does not mean that they cannot properly use logic...they can write Lean-verified theorems. How is that not logic? > They consistently fail at drawing basic logical conclusions because they cannot build a sufficiently abstract model of certain problems that allows the…

This is exactly backwards. The brittleness is because they emulate reasoning without actually algorithmically performing it.

Add.: I pointed to this class of problems specifically because they require the ability to abstract in a way that the question itself does not immediately suggest. Math problems are different in that they are described in terms of art that are closely related to certain patterns of manipulation (that is, the paper texts tend to contain both in close proximity to one another).

Re: Do LLMs pass the mirror test?

#53

Post by jhbadger »

mohsen1 wrote: post #18

It seems like we forget that LLMs are next token prediction systems. Using raw models without instruction following and chat completion bells and whistles will give you a better feeling of what LLMs are. The current interface to LLMs are heavily biased towards "predict the next token in the context of a user with a helpful assistant" but LLMs are capable of other modes of next token prediction too. Before the ChatGPT…

The problem with that argument is it is trivial to write a Markov chain program that takes in text and then can generate the most probable series of words given a starting word. I myself wrote such a program in BASIC on a 64K 8-bit computer in the 1980s after reading one of A.K. Dewdney's columns. That wasn't at all an LLM though. There's a connection, sure, but one that is equating a paper airplane to a jet airliner.

Re: Do LLMs pass the mirror test?

#54

Post by Charon77 »

jhbadger wrote: post #53
mohsen1 wrote: post #18

It seems like we forget that LLMs are next token prediction systems. Using raw models without instruction following and chat completion bells and whistles will give you a better feeling of what LLMs are. The current interface to LLMs are heavily biased towards "predict the next token in the context of a user with a helpful assistant" but LLMs are capable of other modes of next token prediction too. Before the ChatGPT…

The problem with that argument is it is trivial to write a Markov chain program that takes in text and then can generate the most probable series of words given a starting word. I myself wrote such a program in BASIC on a 64K 8-bit computer in the 1980s after reading one of A.K. Dewdney's columns. That wasn't at all an LLM though. There's a connection, sure, but one that is equating a paper airplane to a jet airliner…

The issue with Markov Chain is you can't get good next token prediction on long enough context because once you see the last 1000 words instead of just 2, it's quite unlikely that your 'frequency' is populated for that exact combination, and markov chain don't work on token embedding that allows some encoding of meaning.

Re: Do LLMs pass the mirror test?

#55

Post by linzhangrun »

gmueckl wrote: post #33
post-it wrote: post #30

Earlier quoted context omitted.

> It seems like we forget that LLMs are next token prediction systems. It's pretty clear to me that above a certain size threshold, LLMs are more than a sum of their parts. The sheer amount of training data seems to embed a higher level of reasoning.

There cannot be any reasoning embedded in the model. The algorithm is literally "predict the most likely next token". Anything beyond that is just patterns in the predictions fooling us humans into ascribing more to the system than it is actually producing.

Quantity change leads to quality change. You can check out this Kurzgesagt video on emergence: https://www.youtube.com/watch?v=16W7c0mb-rE

Re: Do LLMs pass the mirror test?

#56

Post by aspenmartin »

gmueckl wrote: post #52
aspenmartin wrote: post #50

Earlier quoted context omitted.

> Reasoning includes things like proper use of logic. LLMs have been repeatedly shown to fail horribly at this. That models cannot do ALL logic problems does not mean that they cannot properly use logic...they can write Lean-verified theorems. How is that not logic? > They consistently fail at drawing basic logical conclusions because they cannot build a sufficiently abstract model of certain problems that allows the…

This is exactly backwards. The brittleness is because they emulate reasoning without actually algorithmically performing it. Add.: I pointed to this class of problems specifically because they require the ability to abstract in a way that the question itself does not immediately suggest. Math problems are different in that they are described in terms of art that are closely related to certain patterns of manipulation…

For you, a system needs to reason perfectly and flawlessly, all the time? So humans do not reason? Humans don't have brittle failure modes?

> they require the ability to abstract in a way that the question itself does not immediately suggest

yes, yet there are multitudes of other measurements of the same kind where LLMs reason perfectly well and better in many cases than a human could.

> Math problems are different in that they are described in terms of art that are closely related to certain patterns of manipulation (that is, the paper texts tend to contain both in close proximity to one another).

Is your logic really that math problems are actually easier to answer without reasoning and just by blending together closely related papers? I would definitely suggest reading the literature a bit more on this topic.

Re: Do LLMs pass the mirror test?

#57

Post by dlcarrier »

vova_hn2 wrote: post #48

> LLMs have seen humans act like conscious beings all over their training data because humans acting like conscious beings IS their training data. How do we know that humans don't learn how to act conscious by observing other humans who act conscious? Consciousness doesn't have a precise definition, but if you ask someone to describe it, there is a good chance that the description will include the concept of internal…

    How do we know that humans don't learn how to act conscious by observing other humans who act conscious?
The magnitude of the self help industry, despite the lack of effectiveness, shows that acting like a human isn't a learned behavior, or we would have figured out how to not act like a human.

Re: Do LLMs pass the mirror test?

#58

Post by etskinner »

gmueckl wrote: post #38
thepasch wrote: post #36

Earlier quoted context omitted.

> The algorithm is literally "predict the most likely next token". That's confusing the training objective with the learned behavior. It's like saying "Stockfish's algorithm is literally 'minimize this number', and therefore, it can't actually play Chess."

Not a valid comparison. Chess algorithms are built around the rules of chess, most notably the turn taking nature of the game (min/max with alpha/beta pruning based on lists of valid moves in any position).

Who's to say the "rules of reasoning" aren't just predicting the next thing that an intelligent person (you) would do? Emergent behavior isn't magic, it's just emergent.

Re: Do LLMs pass the mirror test?

#59

Post by FromTheFirstIn »

thepasch wrote: post #15
adzm wrote: post #14

Earlier quoted context omitted.

It's quite distracting and frustrating. No idea why you'd want the beginning and ends of lines of text to be darker than the center.

Sorry about that, the vignette was mainly meant for the desktop view only but is indeed much more invasive/disruptive in the mobile layout. Should be better now.

Way better- thank you!

Re: Do LLMs pass the mirror test?

#60

Post by applicative »

aspenmartin wrote: post #37
gmueckl wrote: post #33

Earlier quoted context omitted.

There cannot be any reasoning embedded in the model. The algorithm is literally "predict the most likely next token". Anything beyond that is just patterns in the predictions fooling us humans into ascribing more to the system than it is actually producing.

This is just a misconception of how LLMs work and also what reasoning is. “There cannot be any reasoning embedded in the model” a strong statement, what do you mean by reasoning because by any reasonable definition I’m aware of, they clearly are able to exhibit reasoning. The fact that the pre training objective is next token loss has nothing to do with capabilities or their ability to reason. To be highly successful…

LLM output is a kind of dreaming but with the whole of past human text output as dream material. It turns out to be useful if you can direct the hallucination
Post reply on HN