LLMs and self-referentiality • Hacker News Board

Re: LLMs and self-referentiality

#91

Post by fc417fc802 » Fri, Sep 04, 2026, 8:59 PM UTC

DonaldFisk wrote: post #79

fc417fc802 wrote: post #75

Earlier quoted context omitted.

I'm not claiming that any particular disabled person has managed to achieve it. I'm asking on what basis you claim it to be categorically impossible. You've claimed that black swans are impossible, I've asked how you arrived at that conclusion, and your answer is that you looked and don't see any. You're here claiming that what submarines do is somehow fundamentally different than swimming and that we know this becau…

> I'm asking on what basis you claim it to be categorically impossible. Because the neural correlates of consciousness wouldn't exist in their brain. I've even looked into what happens if someone, blind from birth, takes LSD. There aren't any large scale studies as it's very difficult to get approval for them, but the short answer appears to be that they don't see things: https://www.livescience.com/62343-psychedelic…

> Because the neural correlates of consciousness wouldn't exist in their brain.

And what are those in concrete terms? Why can't a computer program have those? You have yet to articulate what special property is giving rise to this.

> somewhere in it there are neurons which correlate with what is being experienced whenever something is looked at.

And what exactly is special about the neuron that the biological hardware can do this but the inorganic can't? Is your claim that current silicon doesn't do this (but future might be able to) or that no silicon construction could _ever_ do this?

You have yet to clearly articulate what this magical pixie dust of consciousness is supposed to be.

Re: LLMs and self-referentiality

#92

Post by fc417fc802 » Fri, Sep 04, 2026, 9:05 PM UTC

DonaldFisk wrote: post #77

fc417fc802 wrote: post #76

Earlier quoted context omitted.

> Consciousness is non=physical and takes that information (after processing in the brain) and directly experiences it. Aren't you inventing that whole cloth? What evidence is there to support this claim? Can you empirically measure consciousness to demonstrate its presence or absence? Is there a lobotomy you can perform to make someone unconscious but still able to respond to those around them? If such a lobotomy is…

> Aren't you inventing that whole cloth? I should have said, "I think that consciousness ..." > So on the one hand you claim it's an extra (ie independent) step ... I should have said, "The conscious experience (of qualia) is an extra step, and if consciousness is fundamental then that's a kind of substance dualism. Thanks for pointing these out - I should have proof-read my reply before posting.

You still haven't answered the question. You think ... why? Based on what?

If consciousness is fundamental (ie substance dualism) then how is it exerting influence over our behavior? Isn't it the equivalent of watching a movie at that point?

If it's not fundamental then what precludes a computer program from exhibiting it?

Re: LLMs and self-referentiality

#93

Post by thatjoeoverthr » Sat, Sep 05, 2026, 8:13 AM UTC

heavenlyblue wrote: post #89

thatjoeoverthr wrote: post #61

Earlier quoted context omitted.

It’s really not like that. If I ask you to tell me about a geographical place I just made up, you can trivially and generally instantly recognize you don’t recognize it. A child can do this. You wouldn’t be able to hold a job or generally get through life without this level of self awareness.

I had a great game as a kid where me and girls were playing for kisses on "words", i.e. "start the next word with the letter of the word I just said". So I invented a whole new vocabulary and then taught them that vocabulary. Apart from the fact that I got a lot of kisses from them they were so happy they learned something new they immediately went to share this with their (and my) parents. Obviously I got a slap but…

No, I’m referring to something else. Not the next word task. Being able to know whether you know something. Next time you encounter a child, ask them something they’re likely to be ignorant of, and see they’re able to say “I don’t know”.

Re: LLMs and self-referentiality

#94

Post by heavenlyblue » Sat, Sep 05, 2026, 9:30 AM UTC

thatjoeoverthr wrote: post #93

heavenlyblue wrote: post #89

Earlier quoted context omitted.

I had a great game as a kid where me and girls were playing for kisses on "words", i.e. "start the next word with the letter of the word I just said". So I invented a whole new vocabulary and then taught them that vocabulary. Apart from the fact that I got a lot of kisses from them they were so happy they learned something new they immediately went to share this with their (and my) parents. Obviously I got a slap but…

No, I’m referring to something else. Not the next word task. Being able to know whether you know something. Next time you encounter a child, ask them something they’re likely to be ignorant of, and see they’re able to say “I don’t know”.

But do you really know? Hence stuff like mansplaining where people vastly overestimate their knowledge compared to average Joe

Re: LLMs and self-referentiality

#95

Post by jwr » Sun, Sep 06, 2026, 9:07 AM UTC

euroderf wrote: post #23

jwr wrote: post #20

I think before we get to self-referentiality, we have a bigger problem to solve, and that's one that I did not expect: namely, at which point do we stop saying that a machine pretends to be conscious and start saying that it is conscious? We have, somewhat unexpectedly, built machines that are very, very good at pretending. Now, I'm not calling the current generation of LLMs we have "conscious," but I can't really de…

Consciousness requires qualia. My 0,02€.

I would (tentatively) agree — but that doesn't invalidate my argument: qualia are not objectively measurable, so you still can't tell if a machine is conscious.