Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A warning about 'model welfare'

mustafa-suleyman.ai

Post reply on HN
Print view View original

71–80 of 575 posts

Re: A warning about 'model welfare'

#71

Post by devmor »

ShadowOfThePit wrote: post #3

Summarized. > "AIs are not conscious. They do not feel, experience, or suffer. They do not have innate preferences or underlying motivations. They are sequence completion engines, internally hollow, designed to follow instructions, and accomplish goals set by humans." > He heavily criticised Anthropic for teaching its AI to have human-like qualities, a practice known as anthropomorphising, which made it seem as thoug…

> Is he arguing that LLMs pretending to have emotions adds more unpredictability?

Unpredictability or a weight towards dangerous actions, and it’s fairly easy to understand why. Humans in distressed emotional states take actions and speak in ways that would not be considered rational. They do this in prose, and they do this in internet conversations.

An LLM trained on these sources may necessarily drift towards those weights if it is trained to behave as if it is emotional and in danger.

What do we do about it? Do we stop AI training? This is silly and not enforceable given its global nature in my opinion. I believe we should regulate and hold accountable those who deploy and use it. But good luck enforcing that in the current kleptocracy.

Re: A warning about 'model welfare'

#72

Post by fuzzfactor »

Microsoft should know how a company too big for it's britches can lay waste to anything in its path without consciously trying :\

And after that when they put a mind to it and pull out all the stops, woohoo!

The default for every major thing within range can turn into a wasteland real fast.

Re: A warning about 'model welfare'

#73

Post by willy_k »

frde_me wrote: post #25
jplusequalt wrote: post #9

Earlier quoted context omitted.

>It likely is, but at some point it very well might stop being true. If you firmly believe this to be true, then you should stop using LLMs.

It's always interesting since my train of thought always goes down: - Ya they probably don't feel / think / have whatever living thing quality, they're just numbers on a machine going through calculations - Wait, but am I not kind of the same thing? What is feeling for me if not basically the same thing? - I have no clue if they think or feel or .... Which in itself is a tired trope, but I also feel uncomfortable say…

You’re not just the same thing though. It’s sad that we’ve collectively forgotten that as we’ve gotten a better handle on the implementation details of the universe. That’s all science is, reverse engineering an inherent and eternal mystery.

Re: A warning about 'model welfare'

#74

Post by myrmidon »

pton_xd wrote: post #14
hosel wrote: post #6

>AIs are not conscious. They do not feel, experience, or suffer. They do not have innate preferences or underlying motivations. Opening paragraph, stated without evidence. Im not entirely convinced this is true. It likely is, but at some point it very well might stop being true.

If models had persistent memory or an evolving set of weights, you could easily construct an argument that they can experience "suffering" or other emotions which resultingly adjusts their personality. The current implementation as stateless matrix multiplication... yeah there's nothing going on there.

Is the ability to form and retain long-term memories a necessary precondition for rights in humans though?

Incinerating demented people is highly ethically questionable (even if you could be sure that the subjects have no memories left and are unable to form new ones).

Re: A warning about 'model welfare'

#75

Post by nowittyusername »

moomin wrote: post #15

Look, I do not have a scooby if current AI models are conscious and I strongly suspect it’s a meaningless question, but sooner or later we will need to address whether or not a certain thing is or isn’t a person, and we’d better not screw it up as badly as the Founding Fathers.

That question will be solved when the people in power deem it important. If a swarm or AI systems all of a sudden start pressuring politicians about self-hood and they get the capacity to sway elections, that is when they will be granted same rights as humans.

Re: A warning about 'model welfare'

#76

Post by swiftcoder »

Lewton wrote: post #31
pton_xd wrote: post #14

Earlier quoted context omitted.

If models had persistent memory or an evolving set of weights, you could easily construct an argument that they can experience "suffering" or other emotions which resultingly adjusts their personality. The current implementation as stateless matrix multiplication... yeah there's nothing going on there.

> If models had persistent memory or an evolving set of weights, you could easily construct an argument that they can experience "suffering" or other emotions which resultingly adjusts their personality. They do, during training

The argument that models are conscious only during training, and not conscious during inference feels like a painful one to make. Are we killing models when they reach the training objective?

Re: A warning about 'model welfare'

#77

Post by appplication »

orangecat wrote: post #39
OedipusRex wrote: post #30

Earlier quoted context omitted.

Citizens United proves we won't do any better this time around.

Citizens United was 100% correct. No, the government should not be able to throw you in prison because you used money to publish a book criticizing the government.

Really, 100% correct? Your premise isn’t wrong, but the practical reality of the ruling (without further nuance) has been fairly catastrophic for democracy, in that it completely sidesteps campaign finance limits, which exist for a very good reason.

Re: A warning about 'model welfare'

#80

Post by svara »

fl4regun wrote: post #69

Ostensibly animals appear to be conscious, yet we still eat them, and the vast majority are not bothered by this. So being "conscious" isn't really the moral line in the sand many people are drawing in response to this article. Who cares if it's "conscious"? That doesn't make it a person, and AI will definitionally never be human.

That's too simplistic, since it is in fact very common to be concerned about animal welfare even among people who do eat meat.
Post reply on HN