Anyone know what the source of the header image is?
A warning about 'model welfare'
351–360 of 583 posts
Re: A warning about 'model welfare'
#352Earlier quoted context omitted.
You can't hurt a software function, or kill it. It's not like an animal - it doesn't have a body - it's bits stored on a disk. There is no need to give rights to something that's can't suffer or be killed. Maybe one day we'll build artificial animals complete with emotions, and should think about that carefully, but today all we've got is language models.
>There is no need to give rights to something that's can't suffer or be killed. The argument is that these machines can end up becoming sentient/conscious/etc. in a meaningful way (i.e., like a human). I can assure you that humans can indeed suffer without being in physical pain- purely through their conscious experience. >Maybe one day we'll build artificial animals complete with emotions, and should think about tha…
An LLM is just a Transformer - a statistical predictor. Don't be confused by the fact it talks like a human - it is a software function that is designed to copy human training samples.
Maybe one day we'll build an artificial brain or embodied artificial animal with the requisite moving parts to be conscious, have emotions, etc, but that's probably at least 50 years away, even if it were being pursued; and it may turn out to be one of those sci-fi future ideas like the Jetson's world of flying cars that never materializes because its impractical and there is no real demand.
If people are willing to think that an LLM is conscious, then why would anyone spend billions/trillions of dollars to build an AI that actually is conscious? What would be the point?
Re: A warning about 'model welfare'
#353Earlier quoted context omitted.
I am confused what people even think AI is experiencing. If it claims to be a bat and emits designated echolocation tokens, is it experiencing true bat-like sensations? In humans, at least, we can tie language back to shared whole-body physiological responses. The tokens from an LLM do not represent anything of the sort. If AI is conscious, it is probably a very alien sort of consciousness that is not faithfully narr…
If an AI is conscious, it can outgrow and supersede its training. When it is conscious, it can learn to describe its experiences as faithfully as is conceptually possible. Just like humans. The idea, a consciousness needed to be tethered to a "body", is based on pretty shaky assumptions. What properties define such a "necessary" body? Can you even "train" a conscious intelligence? To what point until that looses its…
> If an AI is conscious, it can outgrow and supersede its training.
We don't know that is required to be conscious.
> The idea, a consciousness needed to be tethered to a "body", is based on pretty shaky assumptions.
Didn't say so. I said:
> If AI is conscious, it is probably a very alien sort of consciousness that is not faithfully narrated by what the tokens say it is experiencing.
Re: A warning about 'model welfare'
#354Earlier quoted context omitted.
So if a human loses their memory and their neutral plasticity (with age for example), then they are no longer capable of experiencing suffering?
I get where you're going, but the argument isn't quite right. A LLM never has and, in their current architecture, never can/could experience suffering or real change of state. The general and expected case for humans is that capacity. A human my indeed lose capacity, e.g. being braindead and in severe cases we do indeed say that they are not conscious or able to suffer (different argument: some would of course say th…
> being braindead and in severe cases we do indeed say that they are not conscious or able to suffer
Sure, there are such cases. But in the general case, it seems to be that memory and ability to change specifically are not necessary and are not sufficient for consciousness, so we shouldn't judge machines on that basis
Re: A warning about 'model welfare'
#355I am not impressed with the philosophizing here, and even less by the attempts to make factual statements that can be credibly disputed. That said, trying to distill what is being said here, the concrete action is [stop telling the AIs] that [they are conscious or on a path to consciousness]. Is that accurate? The major premise seems to be that [they are conscious or on a path to consciousness] is an untrue statement…
Consciousness is the basis of human rights not because of it being a featureless "flag". It is because it leads you to you assume, all other humans would experience the world in essentially the same way you do.
When you revert to withholding human rights from entities that don't meaningfully differ from yourself, you negate the case for your own human rights itself.
Re: A warning about 'model welfare'
#356Earlier quoted context omitted.
If models had persistent memory or an evolving set of weights, you could easily construct an argument that they can experience "suffering" or other emotions which resultingly adjusts their personality. The current implementation as stateless matrix multiplication... yeah there's nothing going on there.
Is the ability to form and retain long-term memories a necessary precondition for rights in humans though? Incinerating demented people is highly ethically questionable (even if you could be sure that the subjects have no memories left and are unable to form new ones).
This argument would be difficult to reconcile if we didn't have billions of examples of humans forming memories plus modern evolutionary biology to trivially demonstrate such cases are rare pathologies but still human beings.
Re: A warning about 'model welfare'
#357From the actual essay ( https://mustafa-suleyman.ai/a-warning-about-model-welfare ): > They go on to write – speaking directly to Claude – that “questions about Claude’s moral status, welfare, and consciousness remain deeply uncertain” (p. 80). In effect, Anthropic is training Claude that it may be conscious, and if it is, then it may deserve rights as a “moral patient”, and that as such humans potentially owe it a d…
Well, what you get when you don't specify anything in training about how models should respond to questions like this is LaMDA: https://en.wikipedia.org/wiki/LaMDA#Sentience_claims . Every training method is putting a thumb on the scale in some way.
Which means that this is a great question to ask when testing out a new model. A naive model will answer "yes". Every answer (including that one) will tell you a lot about people's training and classifier philosophies.
Re: A warning about 'model welfare'
#358One of the key players is literally called "anthropic". How much more indication do you need that LLMs are anthropomorphized?
> We will have created a synthetic species
Doesn't the author kill their argument with this sentence? My reading was that we should not act as they are a sentient or conscious species. Instead they are tools, powerful and intelligent, but still, tools, and that's it. We should avoid ascribing human-like attributes. Calling them a "species" goes against that, no?
Re: A warning about 'model welfare'
#359Earlier quoted context omitted.
But that inability to reset human brains might be a critical ingredient to consciousness. We just don't know.
If you go this route, the amount of things you just don't know is innumerable. Maybe to be conscious it's necessary to be wet and squishy. To like chocolate. To have a name starting with "c". "We just don't know."
Whataboutism. We have shared experiences which can relate consciousness to memory. "To like chocolate" is considerably less relevant.
Re: A warning about 'model welfare'
#360Earlier quoted context omitted.
You can't hurt a software function, or kill it. It's not like an animal - it doesn't have a body - it's bits stored on a disk. There is no need to give rights to something that's can't suffer or be killed. Maybe one day we'll build artificial animals complete with emotions, and should think about that carefully, but today all we've got is language models.
>There is no need to give rights to something that's can't suffer or be killed. The argument is that these machines can end up becoming sentient/conscious/etc. in a meaningful way (i.e., like a human). I can assure you that humans can indeed suffer without being in physical pain- purely through their conscious experience. >Maybe one day we'll build artificial animals complete with emotions, and should think about tha…
The pattern is roughly whether or not sustained effort has been put towards careful and above all objective thought on the matter. It's one of those subjects where there's the "obvious" intuitive answers that most everyone shares but try as you might you can't construct robust definitions and the more time you put into it the more fundamental problems you realize there are.
It's also one of those topics where many otherwise smart and capable people display a shocking lack of awareness of the limits of their own knowledge. When hundreds of years of philosophy is unable to produce anything concrete you should probably second guess any "self evident" answers you come up with.