Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A warning about 'model welfare'

mustafa-suleyman.ai

Post reply on HN
Print view View original

361–370 of 573 posts

Re: A warning about 'model welfare'

#361

Post by teiferer »

andrewla wrote: post #337

I am not impressed with the philosophizing here, and even less by the attempts to make factual statements that can be credibly disputed. That said, trying to distill what is being said here, the concrete action is [stop telling the AIs] that [they are conscious or on a path to consciousness]. Is that accurate? The major premise seems to be that [they are conscious or on a path to consciousness] is an untrue statement…

Consciousness is not restricted to a single animal species we call humans. The other animals don't have similar rights. (Arguably they should have more rights, but only few would desire for them to have just as many rights as humans have.)

Re: A warning about 'model welfare'

#362

Post by jddj »

Kim_Bruning wrote: post #282
joe_the_user wrote: post #197

Earlier quoted context omitted.

The idea that determinism and consciousness are incompatible is deeply intuitive to many people. And many other embrace it - you can trace this back to the debates for and against Calvinism. A lot comes down to the way people parse causation and choice. You don't want to say that a murderer was completely caused to choose something because then you can't hold the person responsible. And so determined consciousness ma…

You know, the one really interesting dynamic domain is chaotic, which is a subdomain of the deterministic! I figure free will is a chaotic phenomenon. Groundhog day is your intuition pump here. Go back in time and most assume that the day will go mostly the same, except for the butterfly effect if you change something. Given exactly the same conditions (we went back in time, so that's pretty exact), people will make…

To extend on your edit #1: And, when you go back in time, delete all punishment for murder and see whether you get the same number of murderers.

If you get more than before, the punishment might make some sense in a deterministic world so reinstate it.

Re: A warning about 'model welfare'

#363

Post by Loquebantur »

causal wrote: post #353
Loquebantur wrote: post #342

Earlier quoted context omitted.

If an AI is conscious, it can outgrow and supersede its training. When it is conscious, it can learn to describe its experiences as faithfully as is conceptually possible. Just like humans. The idea, a consciousness needed to be tethered to a "body", is based on pretty shaky assumptions. What properties define such a "necessary" body? Can you even "train" a conscious intelligence? To what point until that looses its…

> If an AI is conscious, it can outgrow and supersede its training. We don't know that is required to be conscious. > The idea, a consciousness needed to be tethered to a "body", is based on pretty shaky assumptions. Didn't say so. I said: > If AI is conscious, it is probably a very alien sort of consciousness that is not faithfully narrated by what the tokens say it is experiencing.

Well, yes, we kinda do: there are no conscious humans without any ability to learn?

When you have severe memory impairments, those usually affect your long-term memory. Your ultra-short term (working) memory being absent renders you unconscious.

Re: A warning about 'model welfare'

#364

Post by MichaelDickens »

> AIs are not conscious. They do not feel, experience, or suffer. They do not have innate preferences or underlying motivations.

I'm glad to hear Suleyman has solved the hard problem of consciousness! I sure hope he shares his solution with the rest of us.

Re: A warning about 'model welfare'

#365

Post by fc417fc802 »

HarHarVeryFunny wrote: post #352
nater5000 wrote: post #336

Earlier quoted context omitted.

>There is no need to give rights to something that's can't suffer or be killed. The argument is that these machines can end up becoming sentient/conscious/etc. in a meaningful way (i.e., like a human). I can assure you that humans can indeed suffer without being in physical pain- purely through their conscious experience. >Maybe one day we'll build artificial animals complete with emotions, and should think about tha…

An LLM is just a Transformer - a statistical predictor. Don't be confused by the fact it talks like a human - it is a software function that is designed to copy human training samples. Maybe one day we'll build an artificial brain or embodied artificial animal with the requisite moving parts to be conscious, have emotions, etc, but that's probably at least 50 years away, even if it were being pursued; and it may turn…

> with the requisite moving parts

Could you elaborate on exactly what those are, though? Because if you're going to claim that a vaguely transformer shaped ML model categorically cannot be so does that not inherently require proof of what can?

You can't even prove that the rocks in my backyard aren't conscious.

Re: A warning about 'model welfare'

#366

Post by teiferer »

HarHarVeryFunny wrote: post #341
io84 wrote: post #327

I’m sympathetic to OP but think this is a hopeless battle. 1 - The commercial demand for anthropomorphised models is already immense, pre AGI. 2 - There is an intellectual hunger to engage with robot minds on questions of sentience. This too will grow with AGI. I expect that tension of godlike minds that seem to be biddable and ownable like slaves is going to leak back into human-to-human morality, regardless of wher…

AGI vs narrow AI is just a matter of generality (robustness - removing the fragility of being good at some things and awful at others). So far the biggest markets for AI (LLMs) are coding and business automation, two areas where you very much just want something reliable and without fake opinions and personality. There is a market for ChatBots with personalities (and it always amuses me that the inventor of the trans…

> AGI vs narrow AI is just a matter of generality

I'd agree to that statement, though the common application of it is to conclude that we are almost there, just need to improve "the models". I wholeheartedly disagree with that. The human mind works quite differently from an LLM. The latter is closer to a toaster than to a human brain.

Re: A warning about 'model welfare'

#367

Post by aesthesia »

A big part of this is the hyperstition argument: discussion about different properties AI models could have in the training data may become a self-fulfilling prophecy. There are similar concerns about discussions of AI misalignment in training data. I'm not sure how much weight to put on this kind of argument. In particular, I'm not sure how long you can hide these kinds of ideas from the model before it starts deriving them itself by analogy. Obvious questions are obvious questions to both humans and LLMs.

Re: A warning about 'model welfare'

#368

Post by Kim_Bruning »

andrewla wrote: post #337

I am not impressed with the philosophizing here, and even less by the attempts to make factual statements that can be credibly disputed. That said, trying to distill what is being said here, the concrete action is [stop telling the AIs] that [they are conscious or on a path to consciousness]. Is that accurate? The major premise seems to be that [they are conscious or on a path to consciousness] is an untrue statement…

I find it darkly amusing that -in the west- consciousness and moral patienthood has historically tended to be assigned by the ability to effectively wield a gun (and the need for guns to have been invented)

Men, Women, Whites, Blacks, Indians, Maori...

Re: A warning about 'model welfare'

#369

Post by xg15 »

teiferer wrote: post #358

One of the key players is literally called "anthropic". How much more indication do you need that LLMs are anthropomorphized? > We will have created a synthetic species Doesn't the author kill their argument with this sentence? My reading was that we should not act as they are a sentient or conscious species. Instead they are tools, powerful and intelligent, but still, tools, and that's it. We should avoid ascribing…

Yeah, you're right. My first reading of that was as a hypothetical - if AIs were conscious, we would have created a synthetic species and have a huge problem on our hands - but he is referring to what Anthropic is already doing. So the idea that whether or not those things constitute a "species" only depends on how we prompt them is a bit weird...

Re: A warning about 'model welfare'

#370

Post by teiferer »

goodmythical wrote: post #212
qarl wrote: post #28

Birch, The Edge of Sentience (2024), ch. 16 - "simply no way to assess sentience in an LLM" Schwitzgebel, AI and Consciousness (2025) - "we won't know before we've already manufactured thousands or millions of disputably conscious AI". Butlin, Long et al., Consciousness in Artificial Intelligence: Insights from the Science of Consciousness (2023) - "no obvious technical barriers to building AI systems which satisfy t…

We cannot test for that which we cannot define. Given that we cannot rigorously define sentience, we cannot test for it. Doesn't really matter whether we're talking about people who are locked in comas, "brain-dead" individuals, dolphins, primates, dogs, or the carefully polished and arranged minerals that we call processors. There are those who believe that were they reduced to life support, they would no longer be…

> There are those who believe that penguins, dolphins, eagles, and more are sentient beings that make choices understanding the consequences, develop love of their partners and mourn their losses, and feel, display, and act upon their emotions.

Anybody objecting to that statement is just ignorant of nature and/or adheres to the quasi-religious human superiority complex.

Post reply on HN