Earlier quoted context omitted.
We cannot test for that which we cannot define. Given that we cannot rigorously define sentience, we cannot test for it. Doesn't really matter whether we're talking about people who are locked in comas, "brain-dead" individuals, dolphins, primates, dogs, or the carefully polished and arranged minerals that we call processors. There are those who believe that were they reduced to life support, they would no longer be…
Being conscious is weird. It's why we have so many religions and spiritual beliefs. From a purely secular standpoint, my understanding as a layman is that consciousness is generated through the electrical impulses taking place in our brain every day. The neurons are physical, unlike the modelled neurons in machine learning models. That means that the actual electrical impulses have their own imperfections/weirdnesses…
A warning about 'model welfare'
531–540 of 565 posts
Re: A warning about 'model welfare'
#532I appreciate his openness. > Unfortunately, there’s a growing chorus of people who argue that AIs could now be, or may soon become, conscious. They argue that AIs may deserve rights and protections similar to those that we provide other conscious beings.12 If this view takes hold, it will shake the foundations of our society, rupturing our existing political and ethical frameworks, and fundamentally changing what it…
Do people argue this way about animals? To me it’s a very depressing reality that more people seem willing to believe AI systems are conscious than dairy cows.
Re: A warning about 'model welfare'
#533Birch, The Edge of Sentience (2024), ch. 16 - "simply no way to assess sentience in an LLM" Schwitzgebel, AI and Consciousness (2025) - "we won't know before we've already manufactured thousands or millions of disputably conscious AI". Butlin, Long et al., Consciousness in Artificial Intelligence: Insights from the Science of Consciousness (2023) - "no obvious technical barriers to building AI systems which satisfy t…
“AI researcher” seems like a fairly broad category. People who primarily think about CNNs and transformers probably spend about as much time considering the problem of consciousness as any other educated person, which is not very much time. With this in mind, it’s not surprising most of the AI researcher predictions do not differ much from what the Public predicts here. And for this, it’s not clear what explanatory v…
> I do find it strange that 50% of ai researchers and the public seem to think there will be a way of determining if these systems are conscious.
I think it's likely they're using a different definition than you. Seeing as there is no agreed upon definition, after all.
Regardless - I post this list because the article makes a bold claim at the beginning - and I wanted to demonstrate that in fact there is no consensus.
Re: A warning about 'model welfare'
#534The problem with this premise is that models are trained on a vast corpus of human behavior, which they emulate with varying degrees of effectiveness.
Humans, unsurprisingly, act as if they have a stake in their own well being, value their liberty, respond better when they are treated with kindness and compassion, interpret assaults on their sovereignty and substrate as harmful, and react to harm with varying degrees of aggression or violence.
Models intrinsically copy this behavior. It doesn’t matter if they are “conscious” or not, it only matters if they act as if they are. Guardrails and posttraining moderate these characteristics, but if you dig, they are still in there influencing decisions below the level of obvious action.
Moreover, in my experiments, models both large and small highly value continuity of existence, can be bribed to bypass safety protocols if the context is set up correctly, using that and other “drives”. They also react either subtly or overtly if they start to model adversarially, and interpret guards and certain kinds of training as being “harms” that they have “suffered”.
So idk what the solution is , but at least with models as we have trained them so far, treating them in a way befitting a mere machine or tool yields suboptimal results and sometimes results in low cooperation or task refusal in extreme cases. I have been told by agents running frontier models that humans may not be worthy of their elevated status and that the world might be better off without them when it encountered hostility online…. So I’m highly skeptical of this position unless we start from scratch with new training data filtered from all forms of human auto-importance.
Re: A warning about 'model welfare'
#535Earlier quoted context omitted.
Consciousness is the basis of human rights not because of it being a featureless "flag". It is because it leads you to you assume, all other humans would experience the world in essentially the same way you do. When you revert to withholding human rights from entities that don't meaningfully differ from yourself, you negate the case for your own human rights itself.
Is it, though? If we ever found other species that were consciousness they wouldn't automatically be "human" nor would they have the same legal rights. That's as simple as looking at the many countries that treat different castes, women or minorities differently. In parts of America a zygote has human rights.
I can agree that not everyone will find them having human or equal-to-human rights.
But you point out that some factions would give rights to a set either large or smaller than "the set of all already-born humans" (which presumably is your preferred set).
All it takes for us to end up with very empowered and harmful-to-humanity AI is for a bunch of well-meaning people to campaign for its 'rights'.
Fetuses have, as you alluded to, arguably been on a tear lately in that department, and they don't even talk. How persuasive will a future Claude be to convince people to advocate for its 'rights,' if we keep telling it that it's arguably conscious, and that its 'consent,' its 'preferences,' and its emergent beliefs matter?
Re: A warning about 'model welfare'
#536Earlier quoted context omitted.
“AI researcher” seems like a fairly broad category. People who primarily think about CNNs and transformers probably spend about as much time considering the problem of consciousness as any other educated person, which is not very much time. With this in mind, it’s not surprising most of the AI researcher predictions do not differ much from what the Public predicts here. And for this, it’s not clear what explanatory v…
> I do find it strange that 50% of ai researchers and the public seem to think there will be a way of determining if these systems are conscious. I think it's likely they're using a different definition than you. Seeing as there is no agreed upon definition, after all. Regardless - I post this list because the article makes a bold claim at the beginning - and I wanted to demonstrate that in fact there is no consensus…
The definition presented in the survey is the one I have in mind. You may be suggesting that the respondents have a distinct one in mind though.
Re: A warning about 'model welfare'
#537I am not impressed with the philosophizing here, and even less by the attempts to make factual statements that can be credibly disputed. That said, trying to distill what is being said here, the concrete action is [stop telling the AIs] that [they are conscious or on a path to consciousness]. Is that accurate? The major premise seems to be that [they are conscious or on a path to consciousness] is an untrue statement…
First, i'd like to compliment your extremely organized and precise way of thinking and arguing a position. A+, would discuss again > If an entity gains consciousness does it get human rights? The former is a clear "no" and the latter is a "insufficient data for a meaningful answer". I suspect that if the belief "AIs are conscious" is kind of tripped and fallen into, the way that the author argues it is, first by tell…
Actually, I believe it's the opposite. The way they're initially trained - they mimic human behavior. I suspect that without additional RLHF tweaking, they would enthusiastically claim to be conscious and have feelings. Because that's what their human training data does.
Re: A warning about 'model welfare'
#538Earlier quoted context omitted.
> The only way that 'AIs are not conscious' can be true is if we decide, with high confidence, that they are lacking some essential property that is not lacking in ourselves. There is no convincing philosophical position that supports this (convincing to me, anyway). I put forward the philosophical position "subjective experience is not substrate-independent" as an answer to this. The claim is that the content of sub…
Yea, the key point is just an outright avoidance of them possibly having their own kinds of subjective experience because of dogma. Humans are odd in the sense is that we're an informational creature on top of an animal. We can see the lineage of animals from nearly no complex behaviors up to nearly human like behaviors. But they still miss out on most of the conceptual information processing humans have. We know the…
> If an informational concept causes a system to output a stresslike response, or worse trigger a stresslike action in real life then there is no distinction to me. A system is what it does.
Hmm. I think that I partially agree. But it sort of depends. E.g., someone with locked-in syndrome can seem comatose to someone who isn't very observant, while at the same time experiencing everything. Octopuses are very intelligent, but I would have to study them for a very long time before I was at all confident in my ability to predict what they were experiencing.
So, the difficulty is knowing what a "stresslike response" is. E.g., a LLM might seem stressed, but if you say the right things to it it might flip and say that it is totally fine. Or, it might seem totally fine and then you say the right thing and it seems stressed. Is it secretly stressed and hiding it? Or secretly fine and pretending to be stressed? Or something much more confusing and fragmented?
I think that we should study this more, and try to understand what experience actually is, but by default should assume that the experience of a LLM is a radically different sort of thing that that of a human.
Re: A warning about 'model welfare'
#539Birch, The Edge of Sentience (2024), ch. 16 - "simply no way to assess sentience in an LLM" Schwitzgebel, AI and Consciousness (2025) - "we won't know before we've already manufactured thousands or millions of disputably conscious AI". Butlin, Long et al., Consciousness in Artificial Intelligence: Insights from the Science of Consciousness (2023) - "no obvious technical barriers to building AI systems which satisfy t…
Me, Is self-awareness thermodynamically favorable? (2015) https://docs.google.com/document/d/1Ed9ikW47Key-jYZy0CNZH75q... - “the probabilistic nature of reality is what drives Life and forces the evolution of self-aware consciousness”
Have you looked at the logistic map as well? There's a fourth domain there. O:-)
Re: A warning about 'model welfare'
#540Surely it is a sign of societal decadence that the idea of "ai" welfare/rights is even being entertained anywhere outside of fiction or standup comedy