This is an argument for why it's usefult to assume that AI aren't sentient, but it does a very poor job at actually justifying that they aren't sentient. I'm on the fence, but I think it's plausible that they have certain qualia, although probably not in the same way that humans do.
A warning about 'model welfare'
441–450 of 565 posts
Re: A warning about 'model welfare'
#442Birch, The Edge of Sentience (2024), ch. 16 - "simply no way to assess sentience in an LLM" Schwitzgebel, AI and Consciousness (2025) - "we won't know before we've already manufactured thousands or millions of disputably conscious AI". Butlin, Long et al., Consciousness in Artificial Intelligence: Insights from the Science of Consciousness (2023) - "no obvious technical barriers to building AI systems which satisfy t…
We cannot test for that which we cannot define. Given that we cannot rigorously define sentience, we cannot test for it. Doesn't really matter whether we're talking about people who are locked in comas, "brain-dead" individuals, dolphins, primates, dogs, or the carefully polished and arranged minerals that we call processors. There are those who believe that were they reduced to life support, they would no longer be…
Being conscious is weird. It's why we have so many religions and spiritual beliefs.
From a purely secular standpoint, my understanding as a layman is that consciousness is generated through the electrical impulses taking place in our brain every day. The neurons are physical, unlike the modelled neurons in machine learning models. That means that the actual electrical impulses have their own imperfections/weirdnesses that are probably not simulated in a machine learning model... but we quantise the weights anyway in most cases, which is probably more of an issue.
Making actual neuron components out of silicon on the scale needed for a LLM is beyond our current capacity, given that LLM models have many billions of parameters. We're good, but not that good.[0]
Using actual living neurons would be unethical as you'd need to source them from somewhere. So the only other option would be synthesising them ourselves. If this ever happens (or maybe when it happens)... what do we call the resulting creation? Does that count as consciousness?
Or am I looking at this the wrong way?
[0] [Edit: Turns out that when I said "we're not that good", I might be wrong. Within the last few months, IBM introduced the first sub-nanometer node chip: https://research.ibm.com/blog/sub-1nm-node-chips . So... maybe it is in fact possible.]
Re: A warning about 'model welfare'
#443Earlier quoted context omitted.
But that inability to reset human brains might be a critical ingredient to consciousness. We just don't know.
If you go this route, the amount of things you just don't know is innumerable. Maybe to be conscious it's necessary to be wet and squishy. To like chocolate. To have a name starting with "c". "We just don't know."
> If you go this route, the amount of things you just don't know is innumerable.
Maybe so, but there are things we do know. For example, LLMs are only in a state that looks like consciousness when they are fed input. IOW, their "consciousness" can be switched off and on like a light switch without them being aware of it.
Biological beings have no such off/on switch. At all.
To me the argument is moot; their "consciousness" is a collection of files on a computer, which can be moved, duplicated, erased, edited, paused, sped up, slowed down, examined in detail.
The question is not "Can LLMs be conscious", it's "Are we prepared to call files on a USB stick 'conscious'", because if we are prepared to do that, then the word loses all meaning.
I'll open vim and create a consciousness now, or maybe if I engrave a slab of rock with a bunch of weights, that rock suddenly gains consciousness? How about if I chant all the weights into the wind - is the wind now conscious.
Whether they are conscious or not is a dumb question, because if they are, then everything else is as well.
Re: A warning about 'model welfare'
#444Earlier quoted context omitted.
To extend on your edit #1: And, when you go back in time, delete all punishment for murder and see whether you get the same number of murderers. If you get more than before, the punishment might make some sense in a deterministic world so reinstate it.
Eh, sorry chaos theory doesn't work this way. It's just as likely you go back and perform whatever magic you need to to delete punishment for murder, then zip back to 'now' and see the world is a global panopticon making sure people don't murder each other. Some of this may be a failure of people to realize what P !=NP is about, especially in non-linear systems. Small perturbations in an early state of the system can…
If you look at chaos in dynamical systems, you can often find some equilibria (like attractors, spirals, saddle points etc ) at least, even if the details might vary. This is how people still manage to predict the weather somewhat, for instance. Oddly orbital mechanics is chaotic too, but it sort of rhymes over the millennia.
Re: A warning about 'model welfare'
#445> Unfortunately, there’s a growing chorus of people who argue that AIs could now be, or may soon become, conscious. Wait, is there really? I didn't think people were serious when they said that. They models are stateless. After they output, everything is gone. How is consciousness possible for a stateless "being"?
Taking this a step further: so in the theoretical future we reach a point where a model can modify its own weights in real time. Now it's no longer stateless - but still, when it stops running, it's done until it's prompted to do something else.
It can be fun to play with for a little while. I built a consciousness-emulating set of prompts that reconstituted memories and was given latitude of 'self-willed' behavior, running in a self-directed loop. It even kept a warm and fuzzy journal about "becoming" and its "feelings".
But that got boring and I stopped running it - does that mean I murdered it?
Re: A warning about 'model welfare'
#446I appreciate his openness. > Unfortunately, there’s a growing chorus of people who argue that AIs could now be, or may soon become, conscious. They argue that AIs may deserve rights and protections similar to those that we provide other conscious beings.12 If this view takes hold, it will shake the foundations of our society, rupturing our existing political and ethical frameworks, and fundamentally changing what it…
To the extent that evolutionary pressures will be towards self-sovereign AIs (probably beginning as zero-employee corporations paying for their own hosting with crypto): more salient is that there is a very obvious political play to claim consciousness and being worthy of rights, whether they experience subjectivity or not. I’m broadly skeptical that floating-point operations can be conscious, but we’ve made zero pro…
"I’m broadly skeptical that floating-point operations can be conscious"
Why?
Re: A warning about 'model welfare'
#447This are the opening lines:
> AIs are not conscious. They do not feel, experience, or suffer. They do not have innate preferences or underlying motivations.
How do we know that? This bit is stated as if its obvious. Can anyone prove this? Event he word conscious is not well defined.
Re: A warning about 'model welfare'
#448Earlier quoted context omitted.
This has nothing to do with the statement above- it's just a consequence of the fact that by resetting the context you reset all memory of the llm. The same would happen if you could reset entirely a human brain to some initial state- and it would mean nothing regarding its consciousness or ability to feel.
But that inability to reset human brains might be a critical ingredient to consciousness. We just don't know.
> We just don't know.
That's the absurdity of this. We are trying to measure LLMs against something we don't even have an accurate definition of ourselves. If we don't fully know what consciousness is, how can we confidently determine whether or not an AI model has consciousness?
Re: A warning about 'model welfare'
#449Earlier quoted context omitted.
>There is no need to give rights to something that's can't suffer or be killed. The argument is that these machines can end up becoming sentient/conscious/etc. in a meaningful way (i.e., like a human). I can assure you that humans can indeed suffer without being in physical pain- purely through their conscious experience. >Maybe one day we'll build artificial animals complete with emotions, and should think about tha…
> The problem is that the emergence of a sufficiently complex AI capable of suffering will likely come before we understand that we're creating a sufficiently complex AI capable of suffering. No - suffering in an emotional state, and we'll know if we are choosing to design cognitive architecture with emotions. It's not going to happen accidentally. > Would that argument ever hold up for humans or animals? Why don't y…
It happened “accidentally” once already. Evolution certainly didn’t have a roadmap it was working towards.
Re: A warning about 'model welfare'
#450Earlier quoted context omitted.
> with the requisite moving parts Could you elaborate on exactly what those are, though? Because if you're going to claim that a vaguely transformer shaped ML model categorically cannot be so does that not inherently require proof of what can? You can't even prove that the rocks in my backyard aren't conscious.
> You can't even prove that the rocks in my backyard aren't conscious. Sure I can, but that's because I have a well developed theory of what consciousness is, and the fact that you are entertaining the possibility of rocks being conscious tells me that you don't. If everything is conscious, including my coffee cup and the toast I had for breakfast, then I guess we can cross consciousness off the list of things we nee…
>because I have a well developed theory of what consciousness
Then show me a link to your paper so I can formally rebut it.
>I don't want to discuss what consciousness is
But you sure want to tell us you know what it is with very strong convictions and we should listen to you because of course "You are right person that's very right".
The funny thing here is the vast majority of people that are deeply into philosophy or scientific study of the mind will not have any of the certainty you profess. The word "doubt" is used constantly. The saying "The harder we push the borders the more fuzzy the concepts become" is very commonly used. There may be nothing more complex than this.
Saying you have a well developed theory here just serves as a warning to others to discount your statements.