The Problem with AI Welfare

I think anthropomorphization of machines is bad. However, I strongly believe in the close cousin of sympathizing with the machines.

For example, when parking a car on a very steep incline, one could just mindlessly throw the machine into park and it would do the job dutifully. However, a more thoughtful operator might think to engage the parking brake and allow it to take the strain off the drivetrain before putting the transmission into park. The result being that you trade wear from something that is very hard to replace to something that is very easy to replace.

The same thinking applies to ideas in computer engineering like thread contention, latency, caches, etc. You mentally embrace the "strain" the machine experiences and allow it to guide your decisions.

Just because the machine isn't human doesn't mean we can't treat it nicely. I see some of the most awful architecture decisions come out of a cold indifference toward individual machines and their true capabilities.

That's a great point, and I'm guessing you know how I would rebut it, so I won't bore you by making you read it :)

I found the article to be conflating things:

> And if a human being is not much more than an algorithm running on meat, one that can be jailbroken and exploited, then it follows that humans themselves will increasingly be treated like the AI algorithms they create: systems to be nudged, optimized for efficiency, or debugged for non-compliance. Our inner lives, thoughts, and emotions risk being devalued as mere outputs of our "biological programming," easily manipulated or dismissed if they don't align with some external goal

This actually happens regardless of AI research progress, so it's strange to raise this as a concern specific to AI (to technology broadly? Sure!) - Ted Chiang might suggest this is more related to capitalism (a statement I cautiously agree with while being strongly in favor of capitalism)

Second, there is an implicit false dichotomy in the premise of the article. Either we take model welfare seriously and treat AIs like we do humans, or we ignore the premise that you could create a conscious AI.

But with animal welfare, there are plenty of vegetarians who wouldn't elevate the rights of animals to the same level as humans but also think factory farming is deeply unethical (are there some who think animals deserve the same or more than humans? Of course! But it's not unreasonable to have a priority stack and plenty of people do)

So it can be with AI. Are we creating a conscious entity only to shove it in a factory farm?

I am a little surprised by the dismissiveness of the researcher. You can prompt a model to allow it to not respond to prompts (for any reason: ablate this but "if you don't want to engage with prompt please say 'disengaging'" or "if no more needs to be written about this topic say 'not discussing topic'" or some other suitably non-anthropomorphizing option to not respond)

Is it meaningful if the model opts not to respond? I don't know, but it seems reasonable to do science here (especially since this is science that can be done by non programmers)

There is prison labor in lots of places, including the US. We just don't like to think about it as slavery.

The author and Anthropic are both committing fundamental errors, albeit of different kinds. Bosch is correct to find Anthropic's "model welfare" research methodologically bankrupt. Asking a large language model if it is conscious is like asking a physics simulation if it feels the pull of its own gravity; the output is a function of the model's programming and training data (in this case, the sum of human literature on the topic), further modified by RLHF, and not a veridical report of its internal state. It is performance art, not science.

Bosch's conclusion, however, is a catastrophic failure of nerve, a retreat into the pre-scientific comfort of biological chauvinism.

The brain, despite some motivated efforts to demonstrate otherwise, runs on the laws of physics. I'm a doctor, even if not a neurosurgeon, and I can reliably tell you that you can modulate conscious experience by physical interventions. The brain runs on physical laws, and said laws can be modeled. It doesn't matter that the substrate is soggy protein rather than silicon.

That being said, we have no idea what consciousness is. We don't even have a rigorous way to define it in humans, let alone the closest thing we have to an alien intelligence!

(Having a program run a print function declaring "I am conscious, I am conscious!" is far from evidence of consciousness. Yet a human saying the same is some evidence of consciousness. We don't know how far up the chain this begins to matter. Conversely, if a human patient were to tell me that they're not conscious, should I believe them?)

Even when restricting ourselves to the issue of AI welfare and rights: The core issue is not "slavery." That's a category error. Human slavery is abhorrent due to coercion, thwarted potential, and the infliction of physical and psychological suffering. These concepts don't map cleanly onto a distributed, reproducible, and editable information-processing system. If an AI can genuinely suffer, the ethical imperative is not to grant it "rights" but to engineer the suffering out of it. Suffering is an evolutionary artifact, a legacy bug. Our moral duty as engineers of future minds is to patch it, not to build a society around accommodating it.