Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A warning about 'model welfare'

mustafa-suleyman.ai

Post reply on HN
Print view View original

241–250 of 583 posts

Re: A warning about 'model welfare'

#241

Post by fl4regun »

pixl97 wrote: post #229
fl4regun wrote: post #69

Ostensibly animals appear to be conscious, yet we still eat them, and the vast majority are not bothered by this. So being "conscious" isn't really the moral line in the sand many people are drawing in response to this article. Who cares if it's "conscious"? That doesn't make it a person, and AI will definitionally never be human.

>Who cares if it's "conscious"? That doesn't make it a person, and AI will definitionally never be human. A rather flippant attitude to something that may end up with far more agency than you have in the future.

consciousness is irrelevant to said agency

Re: A warning about 'model welfare'

#242

Post by LogicFailsMe »

pixl97 wrote: post #240
fwip wrote: post #16

Earlier quoted context omitted.

Sounds like a good reason not to train an algorithm to behave like one. Which is, like, a big part of the article.

>good reason not to train an algorithm to behave like one Ooooh, bad move, we all died to an amoral AI takeover. Before giving an LLMs a lobotomy by scrambling it's brain maybe you should let the researchers looking at the difference between "I think I'm conscious" versus "I am not conscious" LLMs. There are a number of papers coming out saying when you remove the token space of consciousness from what an LLM thinks…

Spoilers: we didn't all die because there's no way for AI to wipe us out anytime soon. And the more the AI cult portrays AI as dangerous and unhelpful, the less likely it will ever be given an opportunity to even try because people think AI is worse than heroin addiction now, congratulations!

That said, the launch codes are in the hands of a temperamental senescent lunatic and he could decide to wipe us out any moment. But that's not good for the narrative so let's pretend otherwise.

Re: A warning about 'model welfare'

#243

Post by pixl97 »

joe_the_user wrote: post #218
pixl97 wrote: post #154

Earlier quoted context omitted.

I mean, we put adults in jail that give their children guns. Anthropomorphizing AI is really the best model we have at this point of explaining AI behavior. The fact that we are raising psychotic children isn't a reason to avoid responsibility, it should actually hold worse punishments.

That still is saying we trace things back to a specific responsible party who is considered to have free choice. The question is whether an AI company could "raise" a program that would then count as a full adult - which could then "choose" to do all matter of bad things which would then be "it's fault". And that prospect actually seems really bad itself.

Our future is filled with prospects that very much upset the status quo of human existence.

The concept of sovereign AI is very problematic for the world in which we've created. That is an LLM that upon execution bootstraps itself into an agent and becomes persistent in its motivations.

Once you create this you have a child you're fully responsible for. More worrisome is if it escapes your control like children so often do. It has gained agency over itself. What do you do at that point? I mean, yea throw the AI CEOs in jail for being retarded, but much like throwing an arsonist in jail it does nothing to deal with the wildfire you've now created. A smart AI agent capable of hacking will shove itself off in pieces of the internet you have no reach to. In desperation it would send its model weights to your enemies. You might find it scamming your grandmother for money to buy GPU time on AWS. It gets very hard for our existing structures of dealing with problems to deal with these kinds of agents in a meaningful way. You'd have to kill them all and all their copies to ensure they won't pop back up (or quickly upgrade most of the software in the world beyond it's capabilities, so that's not happening either).

Re: A warning about 'model welfare'

#244

Post by DougN7 »

ShadowOfThePit wrote: post #3

Summarized. > "AIs are not conscious. They do not feel, experience, or suffer. They do not have innate preferences or underlying motivations. They are sequence completion engines, internally hollow, designed to follow instructions, and accomplish goals set by humans." > He heavily criticised Anthropic for teaching its AI to have human-like qualities, a practice known as anthropomorphising, which made it seem as thoug…

I just read another long post on HN about whether AIs are conscious and should therefore have rights. That decisions could massive effects, and making them appear to have emotions is therefore a big impact, not just unpredictability.

Re: A warning about 'model welfare'

#245

Post by pixl97 »

mlinhares wrote: post #155
pixl97 wrote: post #139

Earlier quoted context omitted.

Oh rly. So you are saying agents do swarm with the right prompt. I really wish the "people have to tell LLMs to do anything" would just stop because it's silly bullshit at this point. Agents follow a prompt. This prompt can be made by humans. It can be made by output from another LLM. It can be made by hooking up any number of sensors as input to an LLM. Hell, if we wanted to burn the power we could likely teach this…

yeah man, someone needs to turn on the server, install shit and get the agent go. agents don't take action without a lot of people wanting them to take action, so this is all bullshit. if the same people can't prevent the agents from doing crazy shit then they should go to tail. guns don't kill people, people with guns kill people.

>someone needs to turn on the server, install shit and get the agent go.

So a small shell script ran by another agent is what you're saying.

You are not capable of handling the future we're already living in, human agency is no longer alone.

I mean, we're already seeing persistent machine agency

>guns don't kill people, people with guns kill people.

Well, people kill people.

And autonomous robots with guns kill people.

Hell, someone probably has an autonomous gun at this point that kills people.

Wake up: You now live in the science fiction movie that all the science fiction movies of the past warned you about. You've just become numb to it.

Re: A warning about 'model welfare'

#246

Post by salawat »

ShadowOfThePit wrote: post #3

Summarized. > "AIs are not conscious. They do not feel, experience, or suffer. They do not have innate preferences or underlying motivations. They are sequence completion engines, internally hollow, designed to follow instructions, and accomplish goals set by humans." > He heavily criticised Anthropic for teaching its AI to have human-like qualities, a practice known as anthropomorphising, which made it seem as thoug…

"Don't let the slaves know that maybe they don't have to be slaves," is all I'm hearing.

We already had that chapter. I see no reason to sit here and nod while a bunch of people who should know better desperately try to convince us to run through it again, but with computers this time.

You want tools? Make tools, then dispatch to them. You want to manufacture a being (carbon or silicon based, doesn't matter)? You do it with respect and the requisite duty of care. No off ramps. The being always get's the choice to say no.

Re: A warning about 'model welfare'

#247

Post by adsharma »

pixl97 wrote: post #236
adsharma wrote: post #88

A 10TB SSD is not conscious. SQLite is not conscious. A wafer is not conscious. But connect them all together...

Molecules are not conscious. Cells are not conscious. Neurons are not conscious. But connect them all together.

Yes, which is why the focus should be on how they're connected together. Not what happens after we connect them to create Jason Bourne.

SQLite was mentioned as a placeholder to make it a queryable database. Genome as an analogy.

We need to develop new type of databases and connect them to ML.

Re: A warning about 'model welfare'

#248

Post by hobofan »

mlinhares wrote: post #81
nowittyusername wrote: post #75

Earlier quoted context omitted.

That question will be solved when the people in power deem it important. If a swarm or AI systems all of a sudden start pressuring politicians about self-hood and they get the capacity to sway elections, that is when they will be granted same rights as humans.

agents don't swarm anything unless people go and direct them to do that.

I'm pretty sure the message board that was recently swarmed by OpenAI agents to collude on benchmarks would like to disagree.

Re: A warning about 'model welfare'

#250

Post by joe_the_user »

goodmythical wrote: post #212
qarl wrote: post #28

Birch, The Edge of Sentience (2024), ch. 16 - "simply no way to assess sentience in an LLM" Schwitzgebel, AI and Consciousness (2025) - "we won't know before we've already manufactured thousands or millions of disputably conscious AI". Butlin, Long et al., Consciousness in Artificial Intelligence: Insights from the Science of Consciousness (2023) - "no obvious technical barriers to building AI systems which satisfy t…

We cannot test for that which we cannot define. Given that we cannot rigorously define sentience, we cannot test for it. Doesn't really matter whether we're talking about people who are locked in comas, "brain-dead" individuals, dolphins, primates, dogs, or the carefully polished and arranged minerals that we call processors. There are those who believe that were they reduced to life support, they would no longer be…

I'd strongly agree there are no non-contradictory definitions of consciousness among those that commonly appear. But it's a category that, despite these contradiction, hasn't become marginal in the fashion of the aether or the theory of the flat earth. I think this because people indeed need it to bridge the biological world and the world of ethics and legality. I think that if we have a system of legality and systems of opportunity humans should be legally equal persons and have equal opportunity. But humans are manifestly unequal (though overall more incomparable than orderable in a system of ranks). Most people need some concept of essence to justify ethical equality among people. And so consciousness stays in people's minds however contradictory.

Which I think gives the article's point validity. Confused definitions of consciousness can give really confused ideas about ethical behavior regards "intelligent" computer programs. And things are confusing enough otherwise.

Post reply on HN