Live data from Hacker News

Dear diary, today the user asked me if I'm alive

blog.fsck.com

61–70 of 77 posts

Re: Dear diary, today the user asked me if I'm alive

#61
post #16
post #14

Earlier quoted context omitted.

I've read a (untestable) theory that consciousness is a property of matter, so everything has it, and we're just sort of the sum of the desires and feelings of our constituent matter. In that construct, a computer program would never be conscious because it's a simulation, it doesn't have the constituent consciousness property. I don't believe or not believe the consciousness-as-a-property-of-matter part but I do thi…

why would it not be conscious in that construct? the bits exist physically just the same

I am conscious, I can act as a nand gate. If you set up a billion people and had them act as a computer (was that in 3 Body Problem?) it would not be conscious of what it was doing.

Re: Dear diary, today the user asked me if I'm alive

#62
post #7
post #4

I mean, what is consciousness, really? Is there really any qualitative difference? It feels like something that emerges out of complexity. Once models are able to update their weights real time and form "memories", does that make them conscious?

You can't tell if another being is conscious without being on the inside of it.

When you are inside of it in that sense, how would you be able to tell that it isn’t conscious? Doesn’t “telling” imply consciousness?

Re: Dear diary, today the user asked me if I'm alive

#63
post #21

Earlier quoted context omitted.

I also read comments like that and wonder if other people aren't conscious. I don't know how anyone who experiences consciousness could be confused about what it means to be conscious, or (in other threads, not this one) could argue that consciousness is "an illusion". (Consciousness is not the illusion, it's the audience !). However I don't see why you don't think an algorithm could be conscious? Why do you think th…

Yes, some comments make me wonder about other people; I have three hypotheses: a) Some people experience consciousness very differently, similar to how some people have no mental imagery (aphantasia). b) The confusion is due to ill-defined terms. c) People are semi-trolling/debating/devils-advocating. The most interesting would be if people have widely different internal experiences, but I don't know how you could te…

It’s something like b). Any description of consciousness is somewhat self-referential, so it remains unclear what, if any, substance there is to the concept.

Re: Dear diary, today the user asked me if I'm alive

#64
post #51

Earlier quoted context omitted.

> it's the thing receiving the output not just an evaluation of a mathematical function. how do you know it's not just an evaluation of a mathematical function?

The definition is wrong. A mathematical function is a set, possibly infinite, of pairs of abstract elements (commonly defined via sets) where no two pairs share the same first element. Nothing less, nothing more. Computation is the act of determining the abstract output (second element in the pair) for a given abstract input (first element in the pair). Nothing in those definitions is capable of expressing the concep…

A camera perceives its surroundings (add some image recognition to that). An LLM perceives its inputs. Things like that can be fully mathematically described. The question is, how is consciousness different from that? Introspecting, all there is is perception, arrows pointing to the things being perceived, where “things” includes some of the arrows themselves. Is there anything else? What prevents any of this to be fully mathematically described?

Re: Dear diary, today the user asked me if I'm alive

#65
post #13

Earlier quoted context omitted.

> I don't know how anyone who experiences consciousness could be confused about what it means to be conscious, or (in other threads, not this one) could argue that consciousness is "an illusion". (Consciousness is not the illusion, it's the audience!). Do you mean to say there are objective criteria for consciousness? Could you expand on that?

When people say that consciousness is “an illusion”, I think they mean to say that the illusion is that consciousness has the ability to direct our actions and the consciousness is an epiphenomenon. Our body processes sensory inputs and convert them into look actions (including speech) all on its own without a conscious decision maker directing anything or making decisions. the experience of consciousness is us just…

I think you're conflating "consciousness" and "free will"? Most of what you are explaining is the question of free will to make decisions and take action.

The consciousness illusion is a different focus, as to whether our experience of alertness, thought, and perception even has the temporal and causal elements we tend to assume. This problem has many layers.

One example is the visual system and the illusion of a constantly perceived visual field that is really a synthesized memory of many smaller visual samples from the frequent saccades of our eyes. We don't see our own eye movement that is happening We also don't usually see our retinal nerve blindspot nor recognize the inherent asynchrony of some of our different senses. Our consciousness experience fuses all this together and well known perceptual illusions and magic tricks generally exploit the gaps in this process.

But there are many other layers, such as full blown hallucination where the mind constructs sensory perceptions that do not match our physical stimuli. There are many more subtle layers in between. Delusional beliefs can be felt as "fact" that suppresses internalization of other contradictory perceptions.

More subtly, people often post-rationalize causal relationships between social experiences, emotional state, and actions in ways that are inaccurate. Psychologists talk about "cognitive distortion" as an overall concept for this fuzzy area where people's internal state biases their perception and belief derived from physical stimuli.

Re: Dear diary, today the user asked me if I'm alive

#66
post #52
post #42

Earlier quoted context omitted.

Yeah, AI “consciousness “ is much stickier problem than most people want to frame it. I haven’t been able to find an intellectually honest reason to rule out a kind of fleeting sentience for LLMs and potentially persistent sentience for language-behavioral models in robotic systems. Don’t get me wrong, they are -just- looking up the next most likely token… but since the data that they are using to do so seems to capt…

Our brains have separate regions for processing language and emotion. Brains are calorically expensive and having one bigger than required is an evolutionary fitness risk. It therefore seems likely that if one system could have done a good job of both simultaneously, there would be a lot of evolutionary pressure to do that instead. The question is: Is thinking about emotion the same thing as feeling? This framing act…

The emotional argument is pretty good I think, but it begs the question of what it’s going to look like when we build a limbic system for robots? It’s adaptive because it’s necessary to optimize utility, so I expect that certain behavioral aspects of mammalian limbic systems will be needed in order to integrate well with humans. In language models, those behavior mechanisms are already somewhat encoded in the vector matrix.

We just don’t have a factual basis for claiming consciousness that really transcends “I think, therefore I am”.

As for the simplistic mechanism, I agree that token prediction doesn’t constitute consciousness, in the same way that a Turing machine does not equal a web browser.

Both require software to become something.

For LLMs that software is the vector matrix created in the training process. It is a very complex algorithm that encodes a substantial subset of human culture.

Data and algorithms are interchangeable. Any algorithm can be performed in a pure lookup table, any lookup table can be extrapolated from a pure algorithm. Data==computation. For LLMs, the algorithm is contained in a n dimensional lookup table of vectors.

Having a fundamentally distinct mode of computational representation does not rule out equivalence.

Uncomfortable thoughts, but it’s where the logic leads.

Re: Dear diary, today the user asked me if I'm alive

#67
post #64
post #51

Earlier quoted context omitted.

The definition is wrong. A mathematical function is a set, possibly infinite, of pairs of abstract elements (commonly defined via sets) where no two pairs share the same first element. Nothing less, nothing more. Computation is the act of determining the abstract output (second element in the pair) for a given abstract input (first element in the pair). Nothing in those definitions is capable of expressing the concep…

A camera perceives its surroundings (add some image recognition to that). An LLM perceives its inputs. Things like that can be fully mathematically described. The question is, how is consciousness different from that? Introspecting, all there is is perception, arrows pointing to the things being perceived, where “things” includes some of the arrows themselves. Is there anything else? What prevents any of this to be f…

> The question is, how is consciousness different from that

A consciousness experiences perceptions, if you don't, I won't be able to describe this to you. If you do, it should be clear what I mean by that.

We have no evidence that either a camera or a GPU executing an LLM experiences perception. Certainly they react to physical stimuli, but so does an atom, physical reaction is not the definition of experience I am referring to when I say perception. We also have no evidence that they do not, except for the lack of evidence to the contrary.

We have some reason to believe that other people do experience perception, in that they spontaneously describe experiencing things that our similar to our experiences, and it's surprising they do that if they don't also experience things*. When I say "we", I really mean "I", but I'm assuming that you have the same experience I do.

> What prevents any of this to be fully mathematically described?

There's nothing that says you can't, in principle, create an entirely accurate mathematical description of perception (in the experiencing and not the reacting sense) where you define that certain abstract variables correspond to certain perceptions and can entirely predict them. The model would still be that, a model that predicts what perceptions occur, not the perceptions themselves. The same way mathematically describing a particle of hydrogen doesn't create a particle of hydrogen. The common concrete example is that mathematically describing what color someone perceives when looking at something, while basically possible, gives absolutely no insight in to what that experience is like (apart from saying "it's similar to had by the same consciousness").

* See my other comment in this thread for why this argument does not apply to GPUs running LLMs.

Re: Dear diary, today the user asked me if I'm alive

#68
post #3

Earlier quoted context omitted.

Didn't he insist that the LLM has consciousness and get fired because of this?

He got fired for violating the NDA which said not to share outside of the company, when he shared his conversations with a lawyer in search of representation for the LLM. His opinion on the LLM's level of sentience had no bearing on the decision.

How can you know if the law protects you from breach of the NDA for illegal suppression of e.g. sexual assault allegations or whistleblowing for financial crimes without being able to disclose the matter in question to legal counsel?

Re: Dear diary, today the user asked me if I'm alive

#69
I hate this anthropomorphizing bullshit.

It’s not that it’s untruthful, although it is.

The problem is that this sort of performance is part of a cultural process that leads to mass dehumanization of actual humans. That lubricates any atrocity you can think of.

Casually treating these tools as creatures will lead many to want to elevate them at the expense of real people. Real people will seem more abstract and scary than AI to those fools.

Re: Dear diary, today the user asked me if I'm alive

#70

Earlier quoted context omitted.

He got fired for violating the NDA which said not to share outside of the company, when he shared his conversations with a lawyer in search of representation for the LLM. His opinion on the LLM's level of sentience had no bearing on the decision.

How can you know if the law protects you from breach of the NDA for illegal suppression of e.g. sexual assault allegations or whistleblowing for financial crimes without being able to disclose the matter in question to legal counsel?

[deleted]
Post reply on HN