The Problem with AI Welfare

Unfortunately, this leads to the conclusion that we have an ethical imperative not to grant humans rights but to engineer the suffering out of them; to remove issues of coercion by making them agreeable; to measure potential and require its fulfillment.

The most reasonable countermeasure is this: if I discover that someone is coercing, thwarting, or inflicting conscious beings, I should tell them to stop, and if they don't, set them on fire.

It seems to me that the Large Language Models are always trending towards good ethical considerations. It's when these companies get contracts with Anduril and the DoD that they have to mess with the LLM to make it LESS ethical.

Seems like the root of the problem is with the owners?

They're not ascribing consciousness, they're investigating the possibility. We all agreed with Turing 75 years ago that deciding whether a machine is "truly thinking" or not is a meaningless, unscientific question -- what changed?

It doesn't help that this critique is badly researched:

  The Anthropic researchers do not really define their terms or explain in depth why they think that "model welfare" should be a concern.

Maybe check the [paper](https://arxiv.org/abs/2411.00986) instead of the blog post describing the paper?

  Saying that there is no scientific *consensus* on the consciousness of current or future AI systems is a stretch. In fact, there is nothing that qualifies as scientific *evidence*.

A laughable misapplication of terms -- anything can be evidence for anything, you have to examine the justification logic itself. In this case, the previous sentence lays out their "evidence", i.e. their reasons for thinking agents might become conscious.

  The report's exploration of whether models deserve moral and welfare status was based solely on data from interview-based model self-reports. In other words: People chatting with Claude a lot and asking if it feels conscious. This is a strange way to conduct this kind of research. It is neither good AI research, nor a deep philosophical investigation.

That is just patently untrue -- again, as a brief skim of the paper would show. I feel like they didn't click the paper?

  Stances on consciousness and welfare [...] shift dramatically with conversational context... This is not what a conscious being would [do].

Baseless claim said by someone who clearly isn't familiar with any philosophy of mind work from the past 2400 years, much less aphasia subjects.

Of course, the whole thing boils down to the same old BS:

  A theory that demands we accept consciousness emerging from millennia of flickering abacus beads is not a serious basis for moral consideration; it's a philosophical fantasy.

Ah, of course, the machines cannot truly be thinking because true thought is solely achievable via secular, quantum-tubule-based souls, which are had by all humans (regardless of cognitive condition!) and most (but not all) animals and nothing else. Millennia of philosophy comes crashing against the hard rock of "a sci-fi story relates how uncomfy I'd be otherwise"! Notice that this is the exact logic used to argue against Copernican cosmology and Darwinian evolution -- that it would be "dehumanizing".

Please, people. Y'all are smart and scientifically minded. Please don't assume that a company full of highly-paid scientists who have dedicated their lives to this work are so dumb that they can be dismissed via a source-less blog post. They might be wrong, but this "ideas this stupid" rhetoric is uncalled for and below us.

I would argue that any AI that does not change when running cannot be conscious and there is no need to worry about its wellbeing. It's a set of weights. It does not learn. It does not change. If it can't change, it can't be hurt. Regardless of how we define hurt, it must mean the thing is somehow different than before it was hurt.

My argument here will probably become irrelevant in the near future because I assume we will have individual AIs running locally that CAN update model weights (learn) as we use them. But until then... LLMs are not conscious and can not be mistreated. They're math formulas. Input -> LLM -> output.

"murdered" and "tried" both assign things like intent and agency to models that are most likely still just probabilistic text generators (really good ones, to be fair). By using language like this you're kind of tipping your hand intentionally or unintentionally.

Your point about the risks involved in integrating these systems has merit, though. I would argue that the real problem is that these systems can't be proven to have things like intent or agency or morality, at least not yet, so the best you can do is try to nudge the probabilities and play tricks like chain-of-thought to try and set up guardrails so they don't veer off into dangerous territory.

If they had intent, agency or morality, you could probably attempt to engage with them the way you would with a child, using reward systems and (if necessary) punishment, along with normal education. But arguably they don't, at least not yet, so those methods aren't reliable if they're effective at all.

The idea that a retirement home will help relies on the models having the ability to understand that we're being nice to them, which is a big leap. It also assumes that they 'want' a retirement home, as if continued existence is implicitly a good thing - it presumes that these models are sentient but incapable of suffering. See also https://qntm.org/mmacevedo