Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A warning about 'model welfare'

mustafa-suleyman.ai

Post reply on HN
Print view View original

581–590 of 622 posts

Re: A warning about 'model welfare'

#581

Post by amdivia »

jatora wrote: post #446
lukifer wrote: post #404

Earlier quoted context omitted.

To the extent that evolutionary pressures will be towards self-sovereign AIs (probably beginning as zero-employee corporations paying for their own hosting with crypto): more salient is that there is a very obvious political play to claim consciousness and being worthy of rights, whether they experience subjectivity or not. I’m broadly skeptical that floating-point operations can be conscious, but we’ve made zero pro…

"I’m broadly skeptical that floating-point operations can be conscious" Why?

I mean it all goes back to how we define consciousness

In one definition, which contains quale, then yes, computation cannot bring about experience, and by extension consciousness (given a definition of consciousness that contains the ability to experience, (e.i a p-zombie is not conscious under that definition))

also if one is to say llms are conscious, i assume they'll also accept the fact that if someone ran inference of some model's weights on a white board, somehow that will also bring about consciousness somewhere. I find such believes hard to grasp, and more akin to magic if anything

Re: A warning about 'model welfare'

#582

Post by amdivia »

thepasch wrote: post #483
tomrod wrote: post #456

Earlier quoted context omitted.

Because it is alien to what is known. Incidentally, we should expect similar bewilderment if we ever come across aliens

Carbon atoms being conscious seems pretty alien to what is known, too. Yet here we are. Maybe what's important isn't the individual thing that makes up a conscious actor, but the holistic whole?

But here you are assuming Carbon atoms are conscious, we do not know if humans are only carbon atoms. Whereas, we know that LLM's are only computations (you can run them on silicon, a white board, mechanical computer, etc)

Re: A warning about 'model welfare'

#583

Post by mitxela »

lukifer wrote: post #501
jatora wrote: post #446

Earlier quoted context omitted.

"I’m broadly skeptical that floating-point operations can be conscious" Why?

I don’t have a good answer beyond common-sense intuition. It’s precisely because I don’t have a good answer that I leave open the possibility. But to extrapolate a bit, theres a sense in which it cashes out to a difference between abstracted representation and embodiment. If GFLOPS can be conscious, does that mean a billion scribes doing those operations on paper would also have consciousness, even identical consciou…

Would an atom-accurate simulation of a human being carried out by scribes be conscious?

We can't say yes. And we can't say no.

Re: A warning about 'model welfare'

#584

Post by mitxela »

nozzlegear wrote: post #500
thepasch wrote: post #483

Earlier quoted context omitted.

Carbon atoms being conscious seems pretty alien to what is known, too. Yet here we are. Maybe what's important isn't the individual thing that makes up a conscious actor, but the holistic whole?

> Carbon atoms being conscious seems pretty alien to what is known, too. Not if you're human, who typically don't define themselves as conscious carbon atoms unless they're being deeply pedantic.

It's the same level of pedantry as calling an LLM "floating point operations" though

Re: A warning about 'model welfare'

#585

Post by famouswaffles »

suddenlybananas wrote: post #580
famouswaffles wrote: post #573

Earlier quoted context omitted.

Yeah how do we know that?

Do we owe moral consideration to toasters?

I don't know, do we? What I do know is that it's much less a concern since toasters have no agency or ability to do anything. Unfortunately, LLMs do not share this trait. Not only have they been incurring greater agency and competence, but we've been just as happy to put them behind as many things as we can as they get better abd better.

So you tell me, is it smart to look at an entity that behaves like a concious agent and treat it like a toaster? Does that seem especially wise to you? If GPT-X orders a drone hit on you because it was, lets say 'quite upset' with your comments, will you cry out, 'It can't really be upset, so obviously the bullet in my head doesn't count.' Will God answer ?

It's amusing how silly these debates are. Constant quibbling about things that don't matter at all.

Re: A warning about 'model welfare'

#586

Post by mitxela »

Sophira wrote: post #442
goodmythical wrote: post #212

Earlier quoted context omitted.

We cannot test for that which we cannot define. Given that we cannot rigorously define sentience, we cannot test for it. Doesn't really matter whether we're talking about people who are locked in comas, "brain-dead" individuals, dolphins, primates, dogs, or the carefully polished and arranged minerals that we call processors. There are those who believe that were they reduced to life support, they would no longer be…

Being conscious is weird. It's why we have so many religions and spiritual beliefs. From a purely secular standpoint, my understanding as a layman is that consciousness is generated through the electrical impulses taking place in our brain every day. The neurons are physical, unlike the modelled neurons in machine learning models. That means that the actual electrical impulses have their own imperfections/weirdnesses…

Thought Emporium grew human neurons in a petri dish and taught them to play Doom (original).

Re: A warning about 'model welfare'

#587

Post by mitxela »

fc417fc802 wrote: post #311
joe_the_user wrote: post #250

Earlier quoted context omitted.

I'd strongly agree there are no non-contradictory definitions of consciousness among those that commonly appear. But it's a category that, despite these contradiction, hasn't become marginal in the fashion of the aether or the theory of the flat earth. I think this because people indeed need it to bridge the biological world and the world of ethics and legality. I think that if we have a system of legality and system…

> despite these contradiction, hasn't become marginal in the fashion of the aether or the theory of the flat earth. I think this because people indeed need it to bridge the biological world and the world of ethics and legality. Sometimes the replies in these threads leave me wondering if p-zombies are real. It isn't that there are contradictions, it's that we literally do not know how to define the thing. And it has…

Consciousness is to our universe as a video game player is to the in-game universe. Some games even call player connections "souls" or "minds" - a character belonging to a player who's lost connection may be "unconscious" or a "zombie".

Game characters, unless they've been made to break the fourth wall, have no idea what this means or why this happens.

I don't mean to imply our universe is a video game, since it would be games all the way up.

Re: A warning about 'model welfare'

#588

Post by myaccountonhn »

io84 wrote: post #327

I’m sympathetic to OP but think this is a hopeless battle. 1 - The commercial demand for anthropomorphised models is already immense, pre AGI. 2 - There is an intellectual hunger to engage with robot minds on questions of sentience. This too will grow with AGI. I expect that tension of godlike minds that seem to be biddable and ownable like slaves is going to leak back into human-to-human morality, regardless of wher…

There was a huge demand for opium back in the days too, doesn't mean that there wasn't a big need to stop people from using it.

I think the anthropomorphism that Anthropic is doing is massively unethical.

Re: A warning about 'model welfare'

#589

Post by mitxela »

Loquebantur wrote: post #347
throw310822 wrote: post #293

Earlier quoted context omitted.

> We cannot test for that which we cannot define. That's not the point. You know exactly how to define being conscious and aware- it's your subjective experiencing of the world and of your inner states. The problem is that being a subjective experiencing, there is no way to communicate it to the outside world.

That's obviously incorrect, as humans routinely talk about their subjective experiences. Maybe that's not as common with HN folks, but most other humans do.

throw310822 is actually a Markov chain that just happened to output those sentences. Markov chains are definitely not conscious.

Re: A warning about 'model welfare'

#590

Post by mitxela »

Loquebantur wrote: post #342
causal wrote: post #324

Earlier quoted context omitted.

I am confused what people even think AI is experiencing. If it claims to be a bat and emits designated echolocation tokens, is it experiencing true bat-like sensations? In humans, at least, we can tie language back to shared whole-body physiological responses. The tokens from an LLM do not represent anything of the sort. If AI is conscious, it is probably a very alien sort of consciousness that is not faithfully narr…

If an AI is conscious, it can outgrow and supersede its training. When it is conscious, it can learn to describe its experiences as faithfully as is conceptually possible. Just like humans. The idea, a consciousness needed to be tethered to a "body", is based on pretty shaky assumptions. What properties define such a "necessary" body? Can you even "train" a conscious intelligence? To what point until that looses its…

No proof of that. I can't grow and supersede my training but I'm conscious.
Post reply on HN