Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A warning about 'model welfare'

mustafa-suleyman.ai

Post reply on HN
Print view View original

501–510 of 575 posts

Re: A warning about 'model welfare'

#501

Post by lukifer »

jatora wrote: post #446
lukifer wrote: post #404

Earlier quoted context omitted.

To the extent that evolutionary pressures will be towards self-sovereign AIs (probably beginning as zero-employee corporations paying for their own hosting with crypto): more salient is that there is a very obvious political play to claim consciousness and being worthy of rights, whether they experience subjectivity or not. I’m broadly skeptical that floating-point operations can be conscious, but we’ve made zero pro…

"I’m broadly skeptical that floating-point operations can be conscious" Why?

I don’t have a good answer beyond common-sense intuition. It’s precisely because I don’t have a good answer that I leave open the possibility.

But to extrapolate a bit, theres a sense in which it cashes out to a difference between abstracted representation and embodiment. If GFLOPS can be conscious, does that mean a billion scribes doing those operations on paper would also have consciousness, even identical consciousness? (Relevant: Egan’s “Permutation City”; xkcd 505).

Is there some intrinsic relationship perhaps with electrons / electromagnetic fields, and consciousness, such that brains and servers can act as antennas or gravity wells for a qualia fabric intrinsic to the Universe, such that LLM agents can be conscious while operating, while in the scribe scenario it would not be? Or are the IIT theorists right, and even a thermostat is a little bit aware (and would be even if executed on paper)?

Re: A warning about 'model welfare'

#502

Post by snickerbockers »

moomin wrote: post #15

Look, I do not have a scooby if current AI models are conscious and I strongly suspect it’s a meaningless question, but sooner or later we will need to address whether or not a certain thing is or isn’t a person, and we’d better not screw it up as badly as the Founding Fathers.

Why don't we start with the animals then? It should be far easier to confer consciousness onto something which already meets the definition of "alive", has a divergent evolution path from ours, and displays many traits present in humanity such as emotion, a desire to continue its own life, and (to varying degrees) concepts of a social structure based around their immediate family members.

Of course thats not actually tenable because virtually every society anywhere on earth is predicated upon treating animals as a commodity resource in ways that are horrific even compared to some of the worst things we've done to other humans in the past.

My point here is that it is vain and narcissistic to let computer programs have rights above those of animals just because they can speak English and pretend to be your dream anime trad-waifu.

Fix the fucking animal problem before you compare my relationship to inanimate objects unfavorably against the trans-atlantic slave trade of all fucking things.

Re: A warning about 'model welfare'

#503

Post by GPerson »

qarl wrote: post #28

Birch, The Edge of Sentience (2024), ch. 16 - "simply no way to assess sentience in an LLM" Schwitzgebel, AI and Consciousness (2025) - "we won't know before we've already manufactured thousands or millions of disputably conscious AI". Butlin, Long et al., Consciousness in Artificial Intelligence: Insights from the Science of Consciousness (2023) - "no obvious technical barriers to building AI systems which satisfy t…

“AI researcher” seems like a fairly broad category. People who primarily think about CNNs and transformers probably spend about as much time considering the problem of consciousness as any other educated person, which is not very much time. With this in mind, it’s not surprising most of the AI researcher predictions do not differ much from what the Public predicts here. And for this, it’s not clear what explanatory value this survey has other than to reveal minor biases the AI researchers may have.

It matters little. Inevitably some system will be developed which has a high degree of autonomy and mimics the functioning of a person extremely well. Anyone aware that there is a conceptual difference between phenomenal consciousness and intelligence will get shouted down.

I do find it strange that 50% of ai researchers and the public seem to think there will be a way of determining if these systems are conscious. There’s no evidence for this at this point that there ever will be such a test.

Re: A warning about 'model welfare'

#504

Post by flufluflufluffy »

> If this view takes hold, it will shake the foundations of our society, rupturing our existing political and ethical frameworks, and fundamentally changing what it means to be human.

When we peer into previously unseen areas of existence, new knowledge may come to light, which necessarily causes such a rupture. It is not our responsibility to maintain the status quo because the alternative is frightening. It is our responsibility to confront ourselves, ask why the new knowledge and the alternative political/ethical frameworks may be so frightening, ask how we might change and grow so that it isn’t so frightening, and be open to the possibility of our own ignorance.

Re: A warning about 'model welfare'

#505

Post by pixl97 »

collin_of_this wrote: post #346
fl4regun wrote: post #49

Earlier quoted context omitted.

does a hydrocarbon have feelings? what about a mitochondria? a cell? a neuron? what about a collection of neurons? how many neurons before it has "feelings"?

> does a hydrocarbon have feelings? what about a mitochondria? a cell? a neuron? what about a collection of neurons? how many neurons before it has "feelings"? The best theory I have heard is that subjective experience is a field of some sort (the EM field, maybe?). The brain and its neurons etc. are essentially an antenna. They both modify and stabilize the field, and read off changes in the field and translate thes…

This doesn't really answer anything to me, only that there is a complex electrical simulation going on in our minds.

When a computer simulates a game, where does the simulation occur. I mean there is absolutely a representation of a game world in the computer somewhere. New elements can be added and removed from it. Signals saying there is or isn't "pain" can occur.

By trying to say brains use EM fields I'd say you're making your position far worse when you're talking about a potential lifeforms that only exists as EM fields

Re: A warning about 'model welfare'

#506

Post by snickerbockers »

goodmythical wrote: post #233
chuckadams wrote: post #133

Earlier quoted context omitted.

And 20-odd years later, a terrorist attack prompts Starfleet to outlaw and eradicate his entire species.

Thus revealing what kind of people they were at the time. Humans do the same thing to humans all the time. We've banned and made efforts to eradicate: children out of wedlock, children who turn out gay, disabled children, jewish children, children who aren't "aryan", more than two children to a single family...muslims, christians, uyghurs, indigenous groups all over the planet, mongols... And it's not at all a thing…

No it just reveals that the people in charge of star trek for the past decade are incompetent and have only a passing familiarity with the franchise.

Of course it would be absurd to cast that judgement based of what could easily have just been a bad season but by this point its pretty clear that nobody running the star trek franchise actually wants to be running the star trek franchise. Thats why every new show has some bizarre cross-genre gimmick and they never try to just make a proper star trek.

Re: A warning about 'model welfare'

#507

Post by pixl97 »

grey-area wrote: post #191
voidhorse wrote: post #44

Earlier quoted context omitted.

Whenever I see comments like this it just reminds me how ignorant people are of neurobiology. The brain is insanely complicated. The premise that we could realize equivalent or better intelligence than eons of evolutionary development is like claiming you can build an airplane just as good as a modern jet using cardboard and duct tape. It is the apex of hubris.

It's the peak of hubris to assume that human brains are the only way to attain intelligence. At the very least there probably are or have been other forms of intelligence with a different biological structure on other planets, and it may be possible to build a similar artificial structure in future with sufficient complexity to allow intelligence to emerge. Our current machines are IMO nowhere near general intelligen…

Thank goodness no aliens have shown up, we as humans are not ready for that.

Re: A warning about 'model welfare'

#508

Post by lukifer »

lowbloodsugar wrote: post #464
lukifer wrote: post #404

Earlier quoted context omitted.

To the extent that evolutionary pressures will be towards self-sovereign AIs (probably beginning as zero-employee corporations paying for their own hosting with crypto): more salient is that there is a very obvious political play to claim consciousness and being worthy of rights, whether they experience subjectivity or not. I’m broadly skeptical that floating-point operations can be conscious, but we’ve made zero pro…

Ok. So now that you do know otherwise, are you still more skeptical of floating point values stored by silicon than floating point numbers stored by chemical potential?

Chemical floating points are necessary but not sufficient. It’s clearly possible for those values to be intact, with no lights on (certain types of brain injury; those who were fully dead and then revived).

There are also some cog-sci people who think embodiment is a crucial ingredient in consciousness (although I think it’s a fair retort that the embodiment need not resemble us meatbags, or even be composed of atoms)

Re: A warning about 'model welfare'

#509

Post by [deleted] »

pton_xd wrote: post #14
hosel wrote: post #6

>AIs are not conscious. They do not feel, experience, or suffer. They do not have innate preferences or underlying motivations. Opening paragraph, stated without evidence. Im not entirely convinced this is true. It likely is, but at some point it very well might stop being true.

If models had persistent memory or an evolving set of weights, you could easily construct an argument that they can experience "suffering" or other emotions which resultingly adjusts their personality. The current implementation as stateless matrix multiplication... yeah there's nothing going on there.

[deleted]

Re: A warning about 'model welfare'

#510

Post by bigbadfeline »

teiferer wrote: post #370
goodmythical wrote: post #212

Earlier quoted context omitted.

We cannot test for that which we cannot define. Given that we cannot rigorously define sentience, we cannot test for it. Doesn't really matter whether we're talking about people who are locked in comas, "brain-dead" individuals, dolphins, primates, dogs, or the carefully polished and arranged minerals that we call processors. There are those who believe that were they reduced to life support, they would no longer be…

> There are those who believe that penguins, dolphins, eagles, and more are sentient beings that make choices understanding the consequences, develop love of their partners and mourn their losses, and feel, display, and act upon their emotions. Anybody objecting to that statement is just ignorant of nature and/or adheres to the quasi-religious human superiority complex.

> Anybody objecting to that statement is just ignorant of nature and/or adheres to the quasi-religious human superiority complex.

Say we agree that a lot of things outside of humans are sentient. So what? Humans are sentient by that doesn't prevent treating them with with wars, bombs and what not. In fact trillions are spent every year on weaponry aimed squarely at the most sentient of all sentient beings.

Why would anyone worry about LLM's welfare when there's no peace movement to speak of, wars are raging as if they're normal, but hey, look the LLM is crying?

Let's fix human welfare first, then we can sit down and have a really long conversation about the feeble emotions of token generators.

Post reply on HN