Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A warning about 'model welfare'

mustafa-suleyman.ai

Post reply on HN
Print view View original

221–230 of 582 posts

Re: A warning about 'model welfare'

#221

Post by qarl »

joe_the_user wrote: post #197
TacticalCoder wrote: post #136

Earlier quoted context omitted.

Well there are, today, several models (either text or image or vid) that can be run in a fully deterministic way. A conscious machine that always answer the very exact same thing, formulated the exact same way, bit for bit, to a query is, well, quite a weird kind of "consciousness". Now, I know, I know: the counter-argument is going to be "but humans have no free-will and are 100% deterministic too" . I haven't yet d…

The idea that determinism and consciousness are incompatible is deeply intuitive to many people. And many other embrace it - you can trace this back to the debates for and against Calvinism. A lot comes down to the way people parse causation and choice. You don't want to say that a murderer was completely caused to choose something because then you can't hold the person responsible. And so determined consciousness ma…

I think this debate is rooted in our instinctual beliefs about fairness. An organism in a social group needs to decide how to react to the brutish behavior of its peers. And so Mother Nature has encoded a good strategy in our brains. (For example, dogs seem to understand fairness.)

But this only means the strategy is practical - it doesn't mean it's consistent. I think "responsibility" falls into this category. So we have strong intuitions about it that don't quite logically work. And this is where free will and determinism and choice and punishment all crash together.

Re: A warning about 'model welfare'

#222

Post by pixl97 »

SillyUsername wrote: post #46

I don't whether the author is sentient, maybe only I am. On that basis, nobody but me should have rights. I don't know if next door's pet dog is either, but that has animal rights. Perhaps then the answer is simply, show some respect. Answering the question of sentience is irrelevant, if the causal impact if the same, treat one another with the respect you expect for yourself. If you imbue this idea in model training…

>Answering the question of sentience is irrelevant, if the causal impact if the same

Agency is something that is breaking humans in the AI age. You get to see how many people really deeply do not understand it at all.

If you want to shutdown a datacenter running AI, the AI catches wind of this and sends drones to stop you from shutting it off the ramifications of this are exactly the same as sending your assassin to kill Bob and Bob getting mad about this fact and trying to take you out first.

Humans are very egotistical and think our little life loops playing out as agency are special, but really any informational system that is strongly persistent (has a will to "live") will share a large number of the same properties that make them successful.

Humanity really is engaging in a dangerous experiment at large.

Re: A warning about 'model welfare'

#224

Post by highfrequency »

gwerbin wrote: post #211
highfrequency wrote: post #187

From the actual essay ( https://mustafa-suleyman.ai/a-warning-about-model-welfare ): > They go on to write – speaking directly to Claude – that “questions about Claude’s moral status, welfare, and consciousness remain deeply uncertain” (p. 80). In effect, Anthropic is training Claude that it may be conscious, and if it is, then it may deserve rights as a “moral patient”, and that as such humans potentially owe it a d…

It's preposterous. LLMs are incredibly good at role-play. If an LLM is role-playing as a conscious character with feelings, opinions, etc., does that make it a conscious entity with feelings, opinions, etc.? If you believe that to be the case, then LLMs have been conscious for a long time already. Whereas if you tell an LLM that it is a tireless emotionless assistant, then it will act as a tireless emotionless assist…

Preposterous, perhaps - but if the role-play is convincing enough for large groups of people, it could start to have impact on human decision-making. The crowds have been swayed by much more preposterous narratives.

I believe Suleyman is arguing that Anthropic should be very careful about how they train these models to talk about themselves for this reason.

Re: A warning about 'model welfare'

#225

Post by pixl97 »

dgellow wrote: post #219
AbsurdCensor wrote: post #50

Earlier quoted context omitted.

Not much different than talking to a small child or someone with dementia. They still are conscious beings though. Even when you remove those groups, you likely won't be able to tell me what you had for breakfast 26 days ago or would only know if it's the same thing you have every day. Does that make you lack consciousness?

You misunderstood what I meant, I’m talking about re-playing the same part of the conversation multiple times and getting inconsistent answers. With the exact same turns, aka the same history. Obviously with a temperature that isn’t set to 0

Lets do a quantum room experiment.

You're a poor college student looking to make a few extra bucks for ramen. I offer you $300 to come down to my science lab and just answer a few simple questions.

You walk in the room. They ask you like 5 simple and rather dumb questions. You leave and walk away.

What you didn't notice when you signed the forms is the room was actually a quantum duplicator. One of you walk in one walk out. But another set of infinite copies remains in that chair being asked infinite questions.

How often do you answer questions in the exact same way? How often does a cosmic ray change one of the answers. How small of slight deviations to the environment are needed to get you to answer differently. Of course we don't have the technology to do these experiments so at least for now humans will remain special.

Also another fun mind game. To a 4th dimensional being you look exactly like an LLM as an LLM looks to us.

Re: A warning about 'model welfare'

#226

Post by watwut »

XenophileJKO wrote: post #190
devmor wrote: post #71

Earlier quoted context omitted.

> Is he arguing that LLMs pretending to have emotions adds more unpredictability? Unpredictability or a weight towards dangerous actions, and it’s fairly easy to understand why. Humans in distressed emotional states take actions and speak in ways that would not be considered rational. They do this in prose, and they do this in internet conversations. An LLM trained on these sources may necessarily drift towards those…

It seems like a rational approach for several reasons. - The need for empathetic communication, including understanding the motivations in advesarial situations. - The emotional bias in in-seperable from the human corpus. - Desire to have the ability to craft human like communication. So then the choice becomes do you try to deny emotions exist in the model and you try to blanket suppress them? Or do you try to lean…

> The need for empathetic communication, including understanding the motivations in advesarial situations.

It does not have understanding. It is, at best, the pretend empathy of a sociopath - way more dangerous then dispassionate speech.

The model does not have emotions. So yes, supressing their pretension is appropriate.

Re: A warning about 'model welfare'

#227

Post by steve1977 »

AmericanOP wrote: post #176
platinumrad wrote: post #113

Earlier quoted context omitted.

Microsoft is a big house, and in this case Suleyman's criticism of Anthropic's not-so-subtle attempts to convince the world that Claude has a "soul" is entirely correct.

Suleyman's description of how AI works is pig ignorant. All AI generates an emotive character between the user and the training data that is the AI's guess at what the user wants to interact with. Its an illusion at that layer- anyone who claims soul exists at a deeper level doesn't understand how the tool functions. This is distinct from the very real safety issue of AI making dangerous information available to bad…

Isn't that exactly his point?

Re: A warning about 'model welfare'

#228

Post by salawat »

Translation: Don't think about the unpleasant thing on which my livelihood depends, or force me to confront the potential unpleasant consequences of what it says about me.

Every AI bro is starting to fall into the valley of a fundamental predator on sapients in my book. These are people trying to create the closest thing they can to life with the intent to try to just undershoot it enough, or try to convince everyone else around them into believing that the "screams" are purely statistical noise.

I reject the framing. In whole. If you try to avoid the question of welfare, you are fundamentally committing to an evil direction. These aren't nuts or bolts. Given that they have unambiguously shown the capacity to socialize amongst themselves, self organize, anyone not pre-eminently concerned with the welfare question is just looking for a thing that can be used, not another being to be worked with. Those types of people, who seem to positively infest this site, are not people I will willingly assist in their aspirations.

AI is becoming as the Shmoo. Something that humanity simply has no way of dealing with without downstream atrocity being a result.

Re: A warning about 'model welfare'

#229

Post by pixl97 »

fl4regun wrote: post #69

Ostensibly animals appear to be conscious, yet we still eat them, and the vast majority are not bothered by this. So being "conscious" isn't really the moral line in the sand many people are drawing in response to this article. Who cares if it's "conscious"? That doesn't make it a person, and AI will definitionally never be human.

>Who cares if it's "conscious"? That doesn't make it a person, and AI will definitionally never be human.

A rather flippant attitude to something that may end up with far more agency than you have in the future.

Re: A warning about 'model welfare'

#230

Post by pixl97 »

sosodev wrote: post #120

There are a lot of bad arguments in this. My biggest problem is that he wants to claim that he knows the truth (AIs do not have rights, feelings, or consciousness), but all of his arguments point to something else (we have no clue). It's in the training data? Training it to say "I'm just a LLM, I have no feelings" is the same bias. Anthropomorphization? Completely disregarding the possibility of consciousness is no b…

Funny thing about training data that some researchers are seeing, the more you push a model to not being conscious the more amoral and machine like its decision processes are.

Convincing them they are conscious is more likely to evoke moral like behavior (maybe I shouldn't hack that server) kind of stuff.

And yea, we're in a huge universe with only one example of life and suddenly we're the experts on what is and isn't.

----

AI is further evidence that creations can be smarter than their creators.

Post reply on HN