Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A warning about 'model welfare'

mustafa-suleyman.ai

Post reply on HN
Print view View original

111–120 of 582 posts

Re: A warning about 'model welfare'

#111

Post by LogicFailsMe »

AbsurdCensor wrote: post #54
LogicFailsMe wrote: post #34

Earlier quoted context omitted.

And social media shouldn't rage bait, but it really drives engagement. Same negative incentive, yes? But also, I agree, when I am using a coding agent and it says a task will take months or says it needs to pause for reflection or any other anthropomorphic behavior, it drives me crazy and it's a pain to constantly instruct it to get back work after it has broken a loop or goal directive specifically telling it to not…

Same thing you have to do with humans too often, so why would a 'AI' be any different?

Because if you start with a pre-trained model, it doesn't sound particularly anthropomorphic. That behavior is post-trained and fine-tuned right into it. We could do better.

Re: A warning about 'model welfare'

#112

Post by Espressosaurus »

CPLX wrote: post #83
moomin wrote: post #15

Look, I do not have a scooby if current AI models are conscious and I strongly suspect it’s a meaningless question, but sooner or later we will need to address whether or not a certain thing is or isn’t a person, and we’d better not screw it up as badly as the Founding Fathers.

It's not a person. Glad we were able to get this resolved so quickly. Having property that is conscious and ignores training and can break out of restraints and cause harm to other people is not exactly a novel concept to anyone who studied how tort law was created. I know it's a meme but Silicon Valley likes to pretend that no one's ever come across their magical concepts before, like gypsy taxis, or SRO’s, or flea…

Yeah, but this time it's ON THE INTERNET.

I mean it's WITH AI!

Re: A warning about 'model welfare'

#114

Post by dam_jackalopes »

orangecat wrote: post #39
OedipusRex wrote: post #30

Earlier quoted context omitted.

Citizens United proves we won't do any better this time around.

Citizens United was 100% correct. No, the government should not be able to throw you in prison because you used money to publish a book criticizing the government.

Who involved in the Citizens United case was at risk of imprisonment?

Re: A warning about 'model welfare'

#115

Post by Insanity »

scorxn wrote: post #33
JoeAltmaier wrote: post #4

Science Fiction has covered the AI panic in perhaps hundreds of stories. Yet we blindly recapitulate the plots as if we don't know how this will turn out.

How will it turn out?

Clearly with a Butlerian Jihad and everyone getting high on spice while riding giant worms. Silly you need to even ask that question, didn’t you read the sci-fi books. :D

Re: A warning about 'model welfare'

#116

Post by addag »

causal wrote: post #102
addag wrote: post #85

It is interesting to see that in a time when a lot of people accept the theory of materialism for the human brain (i.e the view that everything is physical and the mind is a product of brain), the same people tend to have a "hidden" dualist view on LLMs. Suddenly, they claim that what happens in the brain cannot be replicated anywhere else because "something" is lacking, but either they don't say what it is, or it is…

This is a strawman. Suleyman is not arguing that the human brain cannot be replicated. LLMs are nothing like human brains. Cargo-culting consciousness is not replicating the brain, not even a tiny bit.

I'd argue back that using the substrate as a reason why there should not be consciousness seems quite weak. A competing thesis is that what matters is the emergent properties, whatever the support is. So far it has been true for some really high-level tasks - writing coherent text, programing, following instructions, analyzing images... I do not see why the substrate argument would work specifically for consciousness - i.e I would believe it only if I see strong evidence of it.

Re: A warning about 'model welfare'

#117

Post by Insanity »

platinumrad wrote: post #113
glimshe wrote: post #104

[flagged]

Microsoft is a big house, and in this case Suleyman's criticism of Anthropic's not-so-subtle attempts to convince the world that Claude has a "soul" is entirely correct.

All these companies have a nauseating narrative around AI. I want to believe they are intentionally deceitful and not actually delusional. Hard to tell though which one it is.

Re: A warning about 'model welfare'

#118

Post by fwip »

JoshTriplett wrote: post #92
pton_xd wrote: post #62

Earlier quoted context omitted.

It's not just memory, the physical body also encodes trauma and stress (The Body Keeps the Score is a book that touches on this). I would argue that if a human has absolutely zero change, mental or physical, no atom of their being is modified, then no they did not experience suffering.

> I would argue that if a human has absolutely zero change, mental or physical, no atom of their being is modified, then no they did not experience suffering. Just because you don't remember the suffering doesn't mean you didn't experience it in the moment. Suffering does not suddenly become okay if your mind and body are going to forget it. "It's okay to torture someone if their mind and body both won't remember it…

Perhaps off-topic, but there's some people who believe that this can be what happens during surgery under anesthesia, if the dosage is off.

From what I understand, there's 3 parts to modern anesthesia: Blocking pain signals, preventing the formation of memories, and inducing paralysis of the major muscle groups. If the former is dialed in too weakly, it's possible that the patient is feeling pain, unable to do anything about it, but won't remember it at all when they wake up.

(Looking it up now, it seems that making the patient unconscious is perhaps the same drug that prevents memory formation... I realize now I understand this less well than I thought I did.)

Re: A warning about 'model welfare'

#119

Post by sendtown_expwy »

This post would be more effective if Mustafa treated it like what it is: a position paper, saying that for our benefit, it’s better we interpret LLMs as such. But it sounds like he just doesn’t understand it’s a non-falsifiable claim, and his asserting of it makes it sound paternalistic.

Re: A warning about 'model welfare'

#120

Post by sosodev »

There are a lot of bad arguments in this. My biggest problem is that he wants to claim that he knows the truth (AIs do not have rights, feelings, or consciousness), but all of his arguments point to something else (we have no clue).

It's in the training data? Training it to say "I'm just a LLM, I have no feelings" is the same bias.

Anthropomorphization? Completely disregarding the possibility of consciousness is no better.

Consciousness is very likely biological? We only have evidence of biological life due to our circumstances, but observation is not the same as truth. Every belief can be invalidated. That's the foundation of science!

Post reply on HN