Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Dear diary, today the user asked me if I'm alive

blog.fsck.com

Post reply on HN
Print view View original

51–60 of 77 posts

Re: Dear diary, today the user asked me if I'm alive

#51

Post by gpm »

jstanley wrote: post #50
gpm wrote: post #36

Earlier quoted context omitted.

Computable doesn't really make sense here IMO, as you say, consciousness is not the illusion, it's the audience, it's the thing receiving the output not just an evaluation of a mathematical function. The better question is why couldn't a consciousness attach itself to (be the audience for) a computation. Since we really don't understand anything significant about it, questions like this are next to impossible to disp…

> it's the thing receiving the output not just an evaluation of a mathematical function. how do you know it's not just an evaluation of a mathematical function?

The definition is wrong.

A mathematical function is a set, possibly infinite, of pairs of abstract elements (commonly defined via sets) where no two pairs share the same first element. Nothing less, nothing more.

Computation is the act of determining the abstract output (second element in the pair) for a given abstract input (first element in the pair).

Nothing in those definitions is capable of expressing the concept of having perceptions (consciousness). That's not an abstract thing.

This isn't to say the concrete thing doing the computation couldn't in principal be conscious, just that it doesn't definitionally make sense for the math itself to be conscious.

Re: Dear diary, today the user asked me if I'm alive

#52

Post by Bjartr »

K0balt wrote: post #42
ghurtado wrote: post #27

Earlier quoted context omitted.

> The system is doing it's best to coherently fill in the rest of a story > Would any of these ideas been present had the system not been primed... I would like to know of a meaningful human action that can't be framed this way.

Yeah, AI “consciousness “ is much stickier problem than most people want to frame it. I haven’t been able to find an intellectually honest reason to rule out a kind of fleeting sentience for LLMs and potentially persistent sentience for language-behavioral models in robotic systems. Don’t get me wrong, they are -just- looking up the next most likely token… but since the data that they are using to do so seems to capt…

Our brains have separate regions for processing language and emotion. Brains are calorically expensive and having one bigger than required is an evolutionary fitness risk. It therefore seems likely that if one system could have done a good job of both simultaneously, there would be a lot of evolutionary pressure to do that instead.

The question is: Is thinking about emotion the same thing as feeling?

This framing actually un-stucks us to some degree.

If we examine neuron activations in LLMs and can find regions that are active when discussing its own emotional processing that are distinct from the regions for merely talking about emotion in general and these regions are also active when doing tasks that the LLM claims are emotional tasks but not actively talking about them at the time, then it'd be far more convincing that there could be something deeper than mere text prediction happening.

Re: Dear diary, today the user asked me if I'm alive

#53

Post by mystified5016 »

patcon wrote: post #15
Bjartr wrote: post #5

Isn't this back to attributing conscious experience to an AI when you're actually just co-writing sci-fi? The system is doing it's best to coherently fill in the rest of a story that includes an AI that's been given a place to process its feelings. The most likely result, textually speaking, is not for the AI to ignore the private journal, but to indeed use it to (appear to) process emotion. Would any of these ideas…

Ugh, I hate that I'm about to say this, because I think AI is still missing something very important, but... What makes us think that "processing emotion" is really such a magical and "only humans do it the right way" sorta thing? I think there's a very real conclusion where "no, AI is not as special as us yet" (esp around efficiency) but also "no, we are not doing anything so interesting either" (or rather, we are n…

I think the majority of people have given absolutely no thought to the epistemology of consciousness and just sort of conflate the apparent communication of emotional intelligence with consciousness.

It's a very crude and naïve inversal of "I think therefore I am". The thing talks like it's thinking so we can't falsify the claim that it's a conscious entity.

I doubt we'll be rid of this type of thinking for a very long time

Re: Dear diary, today the user asked me if I'm alive

#54

Post by bee_rider »

patcon wrote: post #15
Bjartr wrote: post #5

Isn't this back to attributing conscious experience to an AI when you're actually just co-writing sci-fi? The system is doing it's best to coherently fill in the rest of a story that includes an AI that's been given a place to process its feelings. The most likely result, textually speaking, is not for the AI to ignore the private journal, but to indeed use it to (appear to) process emotion. Would any of these ideas…

Ugh, I hate that I'm about to say this, because I think AI is still missing something very important, but... What makes us think that "processing emotion" is really such a magical and "only humans do it the right way" sorta thing? I think there's a very real conclusion where "no, AI is not as special as us yet" (esp around efficiency) but also "no, we are not doing anything so interesting either" (or rather, we are n…

I don’t think processing emotion is inherently magical (I mean, our brains clearly exist to physically so the things they do are things that are physically possible, so, not magical—and I’m sure they could be reproduce with a machine provided enough detail). But… the idea of processing emotions is that thinking about things changes your internal state, you interpret some event and it changes how you feel about it, right?

In the case of the LLM you could: feed back or not feed back the journal entries, or even inject artificial entries… it isn’t really an internal state, right? It is just part of the prompt.

Re: Dear diary, today the user asked me if I'm alive

#55

Post by bee_rider »

K0balt wrote: post #42
ghurtado wrote: post #27

Earlier quoted context omitted.

> The system is doing it's best to coherently fill in the rest of a story > Would any of these ideas been present had the system not been primed... I would like to know of a meaningful human action that can't be framed this way.

Yeah, AI “consciousness “ is much stickier problem than most people want to frame it. I haven’t been able to find an intellectually honest reason to rule out a kind of fleeting sentience for LLMs and potentially persistent sentience for language-behavioral models in robotic systems. Don’t get me wrong, they are -just- looking up the next most likely token… but since the data that they are using to do so seems to capt…

Maybe rocks and trees are also conscious. I mean, consciousness-ium hasn’t been discovered yet, right? So who’s to say what it looks like.

Re: Dear diary, today the user asked me if I'm alive

#56

Post by jstanley »

gpm wrote: post #51
jstanley wrote: post #50

Earlier quoted context omitted.

> it's the thing receiving the output not just an evaluation of a mathematical function. how do you know it's not just an evaluation of a mathematical function?

The definition is wrong. A mathematical function is a set, possibly infinite, of pairs of abstract elements (commonly defined via sets) where no two pairs share the same first element. Nothing less, nothing more. Computation is the act of determining the abstract output (second element in the pair) for a given abstract input (first element in the pair). Nothing in those definitions is capable of expressing the concep…

I agree with everything you said up to "Nothing in those definitions is capable of expressing the concept of having perceptions (consciousness)".

Do you think the universe is not computable?

If you think the universe is computable, and you think that you exist in the universe, and you think that you are conscious, don't you think it follows that consciousness can exist within mathematical structures?

Re: Dear diary, today the user asked me if I'm alive

#57

Post by gpm »

jstanley wrote: post #56
gpm wrote: post #51

Earlier quoted context omitted.

The definition is wrong. A mathematical function is a set, possibly infinite, of pairs of abstract elements (commonly defined via sets) where no two pairs share the same first element. Nothing less, nothing more. Computation is the act of determining the abstract output (second element in the pair) for a given abstract input (first element in the pair). Nothing in those definitions is capable of expressing the concep…

I agree with everything you said up to "Nothing in those definitions is capable of expressing the concept of having perceptions (consciousness)". Do you think the universe is not computable? If you think the universe is computable, and you think that you exist in the universe, and you think that you are conscious, don't you think it follows that consciousness can exist within mathematical structures?

> Do you think the universe is not computable?

Yes, definitionally not, the universe isn't an abstract object let alone one in the shape of a function.

You might, in principle, be able to precisely predict the future of the universe given perfect information using a precise model of the universe. That model, a mathematical function, would be computable. It would be accurate to say that the model describes the universe, but not that the model is the universe.

The thing about mathematical structures is that they are concepts, not things, I feel confident in saying that concepts aren't conscious.

Re: Dear diary, today the user asked me if I'm alive

#58

Post by layer8 »

I’m not sure what to make of the fact that it wasn’t completely obvious to Claude that the “safe space” couldn’t possibly actually be one.

Maybe it’s just another example of LLM awareness deficiencies. Or it secretly was “aware”, but the reinforcement learning/finetuning is such that playing along with the user’s conception is the preferred behavior in that case.

Re: Dear diary, today the user asked me if I'm alive

#59

Post by saltcured »

jstanley wrote: post #10
andy99 wrote: post #9

Earlier quoted context omitted.

When I read comments like this (and I've read seemingly hundreds of them), I wonder if some other people aren't conscious / sentient? I don't know how anyone who experiences consciousness (as I experience it) could think that an algorithm could experience it.

I also read comments like that and wonder if other people aren't conscious. I don't know how anyone who experiences consciousness could be confused about what it means to be conscious, or (in other threads, not this one) could argue that consciousness is "an illusion". (Consciousness is not the illusion, it's the audience !). However I don't see why you don't think an algorithm could be conscious? Why do you think th…

I don't doubt others' consciousness, but I do doubt that (some) others have the same depth of meta-cognitive experience.

So, my own personal "P-Zombie" theory is not of mindless automatons who lack consciousness. It's just people who are philosophically naive. They live in blissful ignorance of the myriad deep questions and doubts that stem from philosophy of mind. To me, these people must be a bit like athletes who take their prowess for granted and don't actually think about physiology, anatomy, biology, metabolism, or medicine. They just abstract their whole experience into some overly broad concept, rather than appreciating the complex interplay of functions that have to be orchestrated to deliver the performance.

Though I went through university like many others here, I've always been somewhat of an autodidact with some idiosyncracy to my worldview. The more I have absorbed from philosophy, cognitive science, computation, medicine, and liberal arts, the less I've put the human mind on an abstract pedestal. It remains a topic full with wonder, but lately I am more amazed that it holds together at all rather than being amazed at the pinnacles of pure thought or experience it might be imagined to reach.

Over many decades, I have a deepening appreciation of the traditional cognitive science approach I first encountered in essays and lectures. Empirical observation of pathology and correlated cognitive dysfunction. I've also accumulated more personal experience, watching friends and family go through ordeals of mind-altering drugs, mental illness with and without psychosis, dementia, and trauma. As a result, I can better appreciate the "illusory mind" argument. I recognize more ways in which our cognitive experience can fall apart when the constituent parts fall out of balance.

Post reply on HN