Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A warning about 'model welfare'

mustafa-suleyman.ai

Post reply on HN
Print view View original

471–480 of 583 posts

Re: A warning about 'model welfare'

#471

Post by lowbloodsugar »

How about three fifths sentient?

We in the US literally fought a war over the this: “They aren’t any better than animals and it will hurt our whole economy if you say otherwise”.

I’m not saying the models are sentient yet. I’m just saying that morality and ethics don’t give us the option of claiming something is non-sentient and has no rights just because it will inconvenience someone economically.

From an entirely selfish perspective, we should be especially careful advancing such opinions when there is a non-zero chance of an armed ASI that looks at us the way we look at ants.

Re: A warning about 'model welfare'

#472

Post by Kim_Bruning »

GrinningFool wrote: post #445
GrinningFool wrote: post #440

> Unfortunately, there’s a growing chorus of people who argue that AIs could now be, or may soon become, conscious. Wait, is there really? I didn't think people were serious when they said that. They models are stateless. After they output, everything is gone. How is consciousness possible for a stateless "being"?

Taking this a step further: so in the theoretical future we reach a point where a model can modify its own weights in real time. Now it's no longer stateless - but still, when it stops running, it's done until it's prompted to do something else. It can be fun to play with for a little while. I built a consciousness-emulating set of prompts that reconstituted memories and was given latitude of 'self-willed' behavior,…

> so in the theoretical future we reach a point where a model can modify its own weights in real time

As an existence proof: We have many types of models that modify their weights in real time. When they stop running, unfortunately we can't restart them again. And we haven't figured out how to duplicate their weights

> But that got boring and I stopped running it - does that mean I murdered it?

I'm on the fence on this.

Ask me again once we have synthetic models with self-modifying weights.

A) You'd then be stopping something unique

B) Self-modifying weights allows for bootstrapping, which means it's likely to increase in ability over time. It'll be an interesting argument cq empirical experiment to see whether the process stops short or exceeds the capabilities of vertebrates. (at which point the moral patienthood question becomes rather more pressing)

Re: A warning about 'model welfare'

#473

Post by alch- »

causal wrote: post #324
qarl wrote: post #28

Birch, The Edge of Sentience (2024), ch. 16 - "simply no way to assess sentience in an LLM" Schwitzgebel, AI and Consciousness (2025) - "we won't know before we've already manufactured thousands or millions of disputably conscious AI". Butlin, Long et al., Consciousness in Artificial Intelligence: Insights from the Science of Consciousness (2023) - "no obvious technical barriers to building AI systems which satisfy t…

I am confused what people even think AI is experiencing. If it claims to be a bat and emits designated echolocation tokens, is it experiencing true bat-like sensations? In humans, at least, we can tie language back to shared whole-body physiological responses. The tokens from an LLM do not represent anything of the sort. If AI is conscious, it is probably a very alien sort of consciousness that is not faithfully narr…

> It can be trained to say it feels like a bat and insist upon it vigorously.

So can you.

Re: A warning about 'model welfare'

#474

Post by adamrezich »

lowbloodsugar wrote: post #464
lukifer wrote: post #404

Earlier quoted context omitted.

To the extent that evolutionary pressures will be towards self-sovereign AIs (probably beginning as zero-employee corporations paying for their own hosting with crypto): more salient is that there is a very obvious political play to claim consciousness and being worthy of rights, whether they experience subjectivity or not. I’m broadly skeptical that floating-point operations can be conscious, but we’ve made zero pro…

Ok. So now that you do know otherwise, are you still more skeptical of floating point values stored by silicon than floating point numbers stored by chemical potential?

TIL IEEE 754 is a fact of nature

Re: A warning about 'model welfare'

#475

Post by GrinningFool »

Kim_Bruning wrote: post #463
GrinningFool wrote: post #440

> Unfortunately, there’s a growing chorus of people who argue that AIs could now be, or may soon become, conscious. Wait, is there really? I didn't think people were serious when they said that. They models are stateless. After they output, everything is gone. How is consciousness possible for a stateless "being"?

Since January people run AI in agent harnesses. In the past months, people have been extending the duration these systems can run autonomously without human intervention. Most recently, some of these long horizon agents (hundreds of them teaming up) have proven themselves quite creative in escaping the sandboxes they were being tested in. They most certainly have a way to retain state, even to the point of deliberate…

All of this is true - yet in between calls of the loop, they don't exist.

Re: A warning about 'model welfare'

#476

Post by dwaltrip »

JW_00000 wrote: post #409
Chance-Device wrote: post #400

Research into model welfare is justified by the mere possibility that we may be manufacturing countless instances of suffering entities. We owe it to them to ensure that we understand and attempt to minimise any suffering they may experience, which requires understanding more about the physical correlates of pain and suffering in order to detect and reduce them.

Chickens experience pain and suffering yet we kill 202 million of them each day. [1] [1] https://ourworldindata.org/how-many-animals-get-slaughtered-...

And how do we stop? Must I become vegan…?

I accept that may be the only meaningful step I personally could take. It’s a large one though…

I do agree with the sibling comment that one terrible thing doesn’t justify another.

Re: A warning about 'model welfare'

#477

Post by qarl »

lelanthran wrote: post #423
qarl wrote: post #138

Earlier quoted context omitted.

> A conscious machine that always answer the very exact same thing, formulated the exact same way, bit for bit, to a query is, well, quite a weird kind of "consciousness". You'll need to explain why.

>> A conscious machine that always answer the very exact same thing, formulated the exact same way, bit for bit, to a query is, well, quite a weird kind of "consciousness". > You'll need to explain why. Where are you going with this? I don't see a conclusion for this line of questioning. I mean, you can't explain why a machine that reliably and predictably produces the same result is "the normal kind of conscious", c…

I ask because I truly don't understand what he's saying. Clearly we have different ideas about consciousness and I have no idea where we diverge.

He seems to be saying that determinism and consciousness are incompatible. But we already have examples that break that rule (us).

Re: A warning about 'model welfare'

#478

Post by HarHarVeryFunny »

pixl97 wrote: post #450
HarHarVeryFunny wrote: post #378

Earlier quoted context omitted.

> You can't even prove that the rocks in my backyard aren't conscious. Sure I can, but that's because I have a well developed theory of what consciousness is, and the fact that you are entertaining the possibility of rocks being conscious tells me that you don't. If everything is conscious, including my coffee cup and the toast I had for breakfast, then I guess we can cross consciousness off the list of things we nee…

>because I have a well developed theory of what consciousness Then show me a link to your paper so I can formally rebut it. >I don't want to discuss what consciousness is But you sure want to tell us you know what it is with very strong convictions and we should listen to you because of course "You are right person that's very right". The funny thing here is the vast majority of people that are deeply into philosophy…

No, there is no reason for you to listen to me.

Go ahead believing rocks are conscious if you like.

Do you go out on weekends asking people to stop abusing rocks?

Rhetorical question - I don't care what you do on weekends.

Bye!

Re: A warning about 'model welfare'

#479

Post by irishcoffee »

fl4regun wrote: post #49
irishcoffee wrote: post #12

Earlier quoted context omitted.

Does a vacuum have feelings? A cellphone? A paper plate? A billion transistors either pulled high or low? That last bit is the point, and no, there are no feelings there.

does a hydrocarbon have feelings? what about a mitochondria? a cell? a neuron? what about a collection of neurons? how many neurons before it has "feelings"?

“No” is the answer you’re asking a question about. Where was this going?
Post reply on HN