Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Do LLMs pass the mirror test?

blog.pascalschuster.de

Post reply on HN
Print view View original

61–70 of 72 posts

Re: Do LLMs pass the mirror test?

#61

Post by applicative »

aspenmartin wrote: post #49
gmueckl wrote: post #44

Earlier quoted context omitted.

What else is there to say? LLMs can at most regurgitate approximations of human reasoning steps in the limited forms in which they may be expressed in the training data or interpolations thereof. That's the core essence of what they are. There is no proper reasoning to be found.

"at most" is wrong. RL with verifiable rewards takes you beyond quality and skills represented in training data, I'm not aware of meaningful fundamental limits here if you scale compute enough even though right now it's highly sample inefficient. Since you refuse to actually define what you consider to be reasoning let me at least put one out there: a system exhibits reasoning when an answer depends on nontrivial int…

There is the obvious limit that human text output is limited. To this you can add the specific testable training that pertains to code, but this degrades the weights for more general communication. Somehow the hype over the successes with coding in the last year or so made everyone forget the intrinsic limit posed by the exhaustion of real human text output, which is absolutely inescapable

Re: Do LLMs pass the mirror test?

#62

Post by gmueckl »

aspenmartin wrote: post #56
gmueckl wrote: post #52

Earlier quoted context omitted.

This is exactly backwards. The brittleness is because they emulate reasoning without actually algorithmically performing it. Add.: I pointed to this class of problems specifically because they require the ability to abstract in a way that the question itself does not immediately suggest. Math problems are different in that they are described in terms of art that are closely related to certain patterns of manipulation…

For you, a system needs to reason perfectly and flawlessly, all the time? So humans do not reason? Humans don't have brittle failure modes? > they require the ability to abstract in a way that the question itself does not immediately suggest yes, yet there are multitudes of other measurements of the same kind where LLMs reason perfectly well and better in many cases than a human could. > Math problems are different i…

Humans are not flawless, but they are much, much better at reasoning than LLMs are. LLMs can be made to fail quite reliably and easily because they cannot build proper manipulatable/predictive models. This is related to the point that Yann LeCunn makes when advocating for world models (for the physical world) with predictive power.

Re: Do LLMs pass the mirror test?

#63

Post by AlecSchueler »

Charon77 wrote: post #54
jhbadger wrote: post #53

Earlier quoted context omitted.

The problem with that argument is it is trivial to write a Markov chain program that takes in text and then can generate the most probable series of words given a starting word. I myself wrote such a program in BASIC on a 64K 8-bit computer in the 1980s after reading one of A.K. Dewdney's columns. That wasn't at all an LLM though. There's a connection, sure, but one that is equating a paper airplane to a jet airliner…

The issue with Markov Chain is you can't get good next token prediction on long enough context because once you see the last 1000 words instead of just 2, it's quite unlikely that your 'frequency' is populated for that exact combination, and markov chain don't work on token embedding that allows some encoding of meaning.

> and markov chain don't work on token embedding that allows some encoding of meaning.

Working on an "encoding of meaning" sure sounds a lot like reasoning.

Re: Do LLMs pass the mirror test?

#64

Post by singularity2001 »

dekdrop wrote: post #19

Why are we asking a language model for a mirror test? Just because it speak like human, have we forget what it is?

Because these models are now also trained on visual data, so they have a common abstract language in the latent space for different kinds of modalities. It's perfectly reasonable to ask the model if it can associate an image of a server with its own existence. In fact it once saw an open process and said "that's me"

Re: Do LLMs pass the mirror test?

#65

Post by DoctorOetker »

micromacrofoot wrote: post #25
Muhammad523 wrote: post #20

> The result was that dogs weren't interested in their unmodified scent in "raw" form, but the modified version was by far the most interesting thing in the room. They spent more time investigating it than any other stimulus in the experiment. I know very well that this is kind of off-topic, and just like the author, i do not claim to know wether dogs (or any other non-human animal for that matter) is self-aware, and…

regardless of knowing his own scent my dog smells his own pee almost every day... sometimes quite intently, it makes me wonder if there's something else going on there

the best consciousness test is observing a creature perform a mirror test to some other kind of creature.

clearly dogs are mirror testing grass somehow, and making sure they don't start growing brains, either by eating some, or breaking the grass leaves by rubbing their back in the grass.

when they roll in other dogs faeces, they are performing a mirror test on their owner..

Re: Do LLMs pass the mirror test?

#66

Post by aspenmartin »

applicative wrote: post #61
aspenmartin wrote: post #49

Earlier quoted context omitted.

"at most" is wrong. RL with verifiable rewards takes you beyond quality and skills represented in training data, I'm not aware of meaningful fundamental limits here if you scale compute enough even though right now it's highly sample inefficient. Since you refuse to actually define what you consider to be reasoning let me at least put one out there: a system exhibits reasoning when an answer depends on nontrivial int…

There is the obvious limit that human text output is limited. To this you can add the specific testable training that pertains to code, but this degrades the weights for more general communication. Somehow the hype over the successes with coding in the last year or so made everyone forget the intrinsic limit posed by the exhaustion of real human text output, which is absolutely inescapable

> To this you can add the specific testable training that pertains to code, but this degrades the weights for more general communication.

Im not sure exactly what you’re saying here, is it that you trade off coding performance and performance on other tasks like communication? If so (correct if not) this isn’t true — generalization happens. Doing good on coding lifts all boats.

> Somehow the hype over the successes with coding in the last year or so made everyone forget the intrinsic limit posed by the exhaustion of real human text output, which is absolutely inescapable

You’re absolutely right that we’re quickly running out of human text data but that isn’t at all the limitation you think it is. No one has “forgotten” this — coding agent performance is primarily from reinforcement learning on synthetic data traces with verifiable rewards, though pretraining is still important.

Also don’t forget: there is a world of multimodal data (video, audio, 3D maps, etc) that is incredibly rich.

Re: Do LLMs pass the mirror test?

#67

Post by arjie »

Interesting. I definitely saw models act strangely when I would swap between models in my harness between rounds. I have a claw where I allowed for each round the model to be probabilistically selected and the results were somewhat worse than when I picked a single model and stuck to it. I blackboxed the whole thing but I should have looked through and seen what the reasoning looked like.

In the end the experiment ended because it doesn’t benefit as much from caching and on-prem inference latency and effective throughput depends a lot on that.

Very cool idea, man. Thanks for sharing.

Re: Do LLMs pass the mirror test?

#68

Post by geon »

post-it wrote: post #30
mohsen1 wrote: post #18

It seems like we forget that LLMs are next token prediction systems. Using raw models without instruction following and chat completion bells and whistles will give you a better feeling of what LLMs are. The current interface to LLMs are heavily biased towards "predict the next token in the context of a user with a helpful assistant" but LLMs are capable of other modes of next token prediction too. Before the ChatGPT…

> It seems like we forget that LLMs are next token prediction systems. It's pretty clear to me that above a certain size threshold, LLMs are more than a sum of their parts. The sheer amount of training data seems to embed a higher level of reasoning.

No. The reasoning is just an illusion. As soon as you look at it closer, it falls apart completely.

https://yosefk.com/blog/llms-arent-world-models.html

https://www.reddit.com/r/programming/comments/1mnc9qf/llms_a...

https://news.ycombinator.com/item?id=44854518

Re: Do LLMs pass the mirror test?

#69

Post by solid_fuel »

post-it wrote: post #30
mohsen1 wrote: post #18

It seems like we forget that LLMs are next token prediction systems. Using raw models without instruction following and chat completion bells and whistles will give you a better feeling of what LLMs are. The current interface to LLMs are heavily biased towards "predict the next token in the context of a user with a helpful assistant" but LLMs are capable of other modes of next token prediction too. Before the ChatGPT…

> It seems like we forget that LLMs are next token prediction systems. It's pretty clear to me that above a certain size threshold, LLMs are more than a sum of their parts. The sheer amount of training data seems to embed a higher level of reasoning.

> It's pretty clear to me that above a certain size threshold, LLMs are more than a sum of their parts. The sheer amount of training data seems to embed a higher level of reasoning.

You're staring at a mirror.

I have never seen any compelling evidence that LLMs actually reason. They are good at predicting the next token, and that is all they do. This action may be useful, but it is not reasoning.

Post reply on HN