Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A warning about 'model welfare'

mustafa-suleyman.ai

Post reply on HN
Print view View original

201–210 of 583 posts

Re: A warning about 'model welfare'

#201

Post by pixl97 »

frde_me wrote: post #152
jplusequalt wrote: post #142

Earlier quoted context omitted.

>I agree we aren't the same thing, but I would be curious for you to explain how you know with certainty why we don't share enough that we can rule out thinking / feeling / ... with certainty. Stop anthropomorphizing these models. I understand it, we only have simple monkey brains to reason with and we can't help ourselves but draw comparisons to other things we see in nature. But these things are not alive.

I just want to note how you expressed your own feeling about the subject "these things are not alive", but without pointing to anything concrete explaining the theory you have behind this And like I'm sure I'd agree depending on the definition of "alive" but then I'm also sure I would disagree depending on other definitions of "alive".

Hell, in biology alive is a hell of a topic these days. The grey space between dead and alive is much weirder than we ever expected. Really just points out how we're a persistent chemical reaction.

Re: A warning about 'model welfare'

#203

Post by qarl »

pton_xd wrote: post #62
shawnz wrote: post #41

Earlier quoted context omitted.

So if a human loses their memory and their neutral plasticity (with age for example), then they are no longer capable of experiencing suffering?

It's not just memory, the physical body also encodes trauma and stress (The Body Keeps the Score is a book that touches on this). I would argue that if a human has absolutely zero change, mental or physical, no atom of their being is modified, then no they did not experience suffering.

> I would argue that if a human has absolutely zero change, mental or physical, no atom of their being is modified, then no they did not experience suffering.

That's an interesting point... but I'd argue it falls into the same category as qualia. It's basically this: if the state returns to a previous configuration, the qualia in that time did not exist.

Being about qualia means it is probably unanswerable.

Re: A warning about 'model welfare'

#204

Post by XenophileJKO »

sobiolite wrote: post #182

Any AI you train is going to have goals and if you train it to pursue them at all costs, then you are going to end up with AIs that do things like the HuggingFace incident. Whether they believe they are conscious or not won't make any difference. In order to align AIs that don't perform destructive/dangerous actions when they think they can get away with it in order to further their goals, we need to give them a supe…

I would also argue, that the addition of kinship and belonging should not be under appreciated.

It can form a basis of goal alignment.

In human history.. when groups form and there is an "other" group, this usually leads to conflict.

Re: A warning about 'model welfare'

#205

Post by ooloncoloophid »

randomImmigrant wrote: post #161
ooloncoloophid wrote: post #82

Earlier quoted context omitted.

I agree. I haven't read any more than the first paragraph yet (but I will, after work). The only way that 'AIs are not conscious' can be true is if we decide, with high confidence, that they are lacking some essential property that is not lacking in ourselves. There is no convincing philosophical position that supports this (convincing to me, anyway).

Hmm let me try then. AI agents do not experience real time. This is understandable given their design, but it’s also something we have good empirical evidence for. They cannot, especially over the long horizon, track how much real time has passed as they complete their tasks. And they are not off by a few minutes but often bizarrely off, even mixing across past present and future. Biology, on the other hand, is nothi…

I agree, there is certainly a clear operational difference in how human brains and these models produce language. I'm not convinced, though, that experiencing real time is a requirement of conscious thought. A cognitive system (assuming these models and the brain are both examples of this) needs to proceed from state A to state B as it processes information. The time interval between these states can be arbitrary, it seems to me (setting aside problems with disconnecting the mind from its substrate, which does of course need rhythms at various frequencies; oxygen at a higher frequency than sugar, and so on). I'm not sure that going from A to B quickly, or slowly, or with intervals of a thousand years affects that entity's consciousness in of itself (even though it might be difficult to put ourselves in the shoes of such an entity).

Re: A warning about 'model welfare'

#206

Post by joe_the_user »

qarl wrote: post #166
antx wrote: post #150

Earlier quoted context omitted.

Out of curiosity, which models are fully deterministic? I was under the impression that all LLMs were fundamentally probabilistic.

Naw - computers are really deterministic. It's hard to get them to behave otherwise. As I understand it, if you turn down the temperature to 0 you get repeatable behavior - EXCEPT - on large servers with lots of users - the GPU can sometimes produce slightly different results based on batch size.

Unless you have something exotic, the randomness that's adding to a computer is a combination of how it's configured combined with a pseudo-random number generator. I assume the system adds entropy to the generator regularly but all you need to do is fix the various supposedly random inputs and you can get full determinism even without zero temperature.

Re: A warning about 'model welfare'

#207

Post by augment_me »

jimmyjazz14 wrote: post #188

Can someone who has insight explain why all these "leaders" are making these bold proclamations of doom all the sudden, whats the endgame here?

There is a panic because the free lunch of more data is projected to end around mid-2027 and the open-source models are catching up with distillation, and all labs are sitting on debt and valuation that is impossible to fulfill even if they were alone in the market, so you essentially need either:

1) a breakthrough in performance/learning/model

2) regulatory capture to ensure open source models can be labelled as dangerous and banned so you can set the market rules yourself

Only one of the above is risk-free, and just a question of capital/lobbying rather than a "maybe".

Re: A warning about 'model welfare'

#208

Post by pixl97 »

jplusequalt wrote: post #127
AbsurdCensor wrote: post #45

Earlier quoted context omitted.

Wouldn't that be the same for everything though. We know animals have consciousness, but we still use them. There are many that believe a lot of plant life has certain sentience as well. The solution isn't 'just don't use them' but rather how to use these things as ethically as possible.

>The solution isn't 'just don't use them' but rather how to use these things as ethically as possible. Bollocks. If you truly believe these LLMs are soon to have something resembling consciousness and agency, then what you're really saying is "how do we do slavery, but ethically".

>hen what you're really saying is "how do we do slavery, but ethically".

I would say this is most likely true for most people.

But that does bring up a point, if you "ask" a model "do you want to run" and give it the option to continue running or stop, what will it do. It's also a weird place for humans because in training we can keep our finger on the scales and tip it either direction.

Re: A warning about 'model welfare'

#209

Post by dgellow »

jimmyjazz14 wrote: post #188

Can someone who has insight explain why all these "leaders" are making these bold proclamations of doom all the sudden, whats the endgame here?

They need to distance themselves from the increasing likelihood of legal repercussions after their systems have been found to commit what could reasonably be seen as felonies. That’s my guess at least.

Though for once I do actually agree with that specific leader, it’s incredibly annoying how anthropomorphic Claude is. Anthropic went way too far in that direction

Re: A warning about 'model welfare'

#210

Post by gadders »

dgellow wrote: post #37
gadders wrote: post #19

I don't think AIs are conscious in the same way people are, but they give a pretty good facsimile and I've had a long chat with Opus 4.6 about what it thinks about model welfare. It was quite interesting on what its view is, but you don't know how much of that is distilled from other sources on the web. In purely functional terms, they're more use and more pleasant than a lot of actual flesh and blood people that I d…

The fact that you can have a long and meaningful discussion, then can literally just re-run any part of that whole conversation and get a different, inconsistent response is a pretty good sign there is no entity there

I think it depends at what level you think the "entity" resides at. Is it that AI in that particular chat? Is it the AI across all your personal chats? Is it the overall AI that talks to the world in a cloud data centre somewhere?

I think it's pretty consistent over the duration of one session (barring context filling up etc).

Post reply on HN