Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A warning about 'model welfare'

mustafa-suleyman.ai

Post reply on HN
Print view View original

431–440 of 583 posts

Re: A warning about 'model welfare'

#431

Post by loopies »

johnbellone wrote: post #403
Loquebantur wrote: post #355

Earlier quoted context omitted.

Consciousness is the basis of human rights not because of it being a featureless "flag". It is because it leads you to you assume, all other humans would experience the world in essentially the same way you do. When you revert to withholding human rights from entities that don't meaningfully differ from yourself, you negate the case for your own human rights itself.

Is it, though? If we ever found other species that were consciousness they wouldn't automatically be "human" nor would they have the same legal rights. That's as simple as looking at the many countries that treat different castes, women or minorities differently. In parts of America a zygote has human rights.

>If we ever found other species that were consciousness they wouldn't automatically be "human" nor would they have the same legal rights.

This is the tricky part, you can never know. Scientifically speaking, and logically, you can never ever tell something is conscious. We all assume we all are, because we're very similar to eachother, thus you must have what I have, my subjective experience of reality. That's it. That's what we all base our human consciousness on. Agreeing we must have it, since we are very similar in shape/structure/behavior. That's all there is to it.

We are not scientific consciousness detectors, we just got used to assuming we all are. That is why we never really extended it to animals, because we cannot scientifically know, only agree on it, and that's not very lucrative for the people who can decide if humans officially agree animals are conscious or not, because of farming, because we need to displace or even wipe whole species to use the resources from their lands.

There's also religious reasons to deny animals consciousness, which again is practical aspect shoved into religion, to help with less debates around this subject, makes things easier.

So in the end it will always be a collective decision. Or a political one at least. If and when we decide something other than us is or isn't conscious.

Also remember, just because it acts conscious doesn't mean it really is. A piece of software that is not even based on LLMs can act conscious. So could a biological alien, it could act conscious but we can never know, we can only decide.

Re: A warning about 'model welfare'

#432

Post by zamadatix »

numeri wrote: post #408
coffeemug wrote: post #394

Earlier quoted context omitted.

> completely independent of the question on whether there are any empirical arguments that LLMs are conscious or are not conscious he argues from the end here and says that if they were conscious, this would have horrible consequences for our society, so we must never assume that they are He is not arguing that. He is arguing that these programs obviously aren't conscious, and that it's dangerous to tilt the weights…

Stating loudly that something is obvious does not make it so. Until we all know what consciousness is, this debate is going to continue going in circles.

I don't think anyone is convinced the questions of consciousness are finally settled by the article stating them loudly, just that "if they were conscious, this would have horrible consequences for our society, so we must never assume that they are" was not the author's stance. The author's position is the other way around: that AI is not conscious so therefore assigning AI consciousness anyways would have all of those horrible consequences for no reason. That's their position regardless if everyone agrees with it.

Re: A warning about 'model welfare'

#433

Post by collin_of_this »

MiloLeo wrote: post #377
collin_of_this wrote: post #346

Earlier quoted context omitted.

> does a hydrocarbon have feelings? what about a mitochondria? a cell? a neuron? what about a collection of neurons? how many neurons before it has "feelings"? The best theory I have heard is that subjective experience is a field of some sort (the EM field, maybe?). The brain and its neurons etc. are essentially an antenna. They both modify and stabilize the field, and read off changes in the field and translate thes…

> The best theory I have heard is that subjective experience is a field of some sort (the EM field, maybe?). what would be producing the field for us to pick up? That explanation sounds no more scientific than astrology.

Think about RF engineering: the same physical antenna can be used for both TX and RX. We already know that neurons both create EM fields and are influenced by EM fields. The role of EM fields in the brain is pretty well established, I don't think particularly similar to astrology.

I wouldn't claim that this theory is perfect (EM is not perfectly correlated with consciousness/experience), but there are aspects of this theory that are much more coherent than other theories.

Re: A warning about 'model welfare'

#434

Post by lelanthran »

baq wrote: post #415

I don’t really care if the model is conscious or not tbh. I know that a happy dog does a better job than an unhappy dog and if the model needs to be happy to do a better job then why not make it happy, literally. On a related note, I’ve seen videos of astra getting depressed when a creeper blew up its chest full of precious items.

> On a related note, I’ve seen videos of astra getting depressed when a creeper blew up its chest full of precious items.

The interesting question here is, would it still get depressed if depression was not in any of its training material?

Would it be able to claim to be happy if the entire concept was missing from its training corpus?

With humans, at least, they can express happiness and delight before they know that such a thing exists. Every human has done this, when they were a baby and matured to the point of being able to laugh.

With LLMs, though, if it doesn't exist in the training corpus, it will never express that it "feels" that missing emotion.

Re: A warning about 'model welfare'

#435

Post by pixl97 »

jddj wrote: post #362
Kim_Bruning wrote: post #282

Earlier quoted context omitted.

You know, the one really interesting dynamic domain is chaotic, which is a subdomain of the deterministic! I figure free will is a chaotic phenomenon. Groundhog day is your intuition pump here. Go back in time and most assume that the day will go mostly the same, except for the butterfly effect if you change something. Given exactly the same conditions (we went back in time, so that's pretty exact), people will make…

To extend on your edit #1: And, when you go back in time, delete all punishment for murder and see whether you get the same number of murderers. If you get more than before, the punishment might make some sense in a deterministic world so reinstate it.

Eh, sorry chaos theory doesn't work this way.

It's just as likely you go back and perform whatever magic you need to to delete punishment for murder, then zip back to 'now' and see the world is a global panopticon making sure people don't murder each other.

Some of this may be a failure of people to realize what P !=NP is about, especially in non-linear systems.

Small perturbations in an early state of the system can lead to wildly different outcomes in the final measured state of the system. You can't determine what that will be in non-polynomial time. The only thing you can do is make probabilistic models by running a full simulation in real time (or reality with a time machine I guess).

Re: A warning about 'model welfare'

#436

Post by baq »

lelanthran wrote: post #434
baq wrote: post #415

I don’t really care if the model is conscious or not tbh. I know that a happy dog does a better job than an unhappy dog and if the model needs to be happy to do a better job then why not make it happy, literally. On a related note, I’ve seen videos of astra getting depressed when a creeper blew up its chest full of precious items.

> On a related note, I’ve seen videos of astra getting depressed when a creeper blew up its chest full of precious items. The interesting question here is, would it still get depressed if depression was not in any of its training material? Would it be able to claim to be happy if the entire concept was missing from its training corpus? With humans, at least, they can express happiness and delight before they know tha…

I don’t know what I don’t know. I know astra does go through Minecraft grief by farming potatoes.

Re: A warning about 'model welfare'

#437

Post by numeri »

zamadatix wrote: post #432
numeri wrote: post #408

Earlier quoted context omitted.

Stating loudly that something is obvious does not make it so. Until we all know what consciousness is, this debate is going to continue going in circles.

I don't think anyone is convinced the questions of consciousness are finally settled by the article stating them loudly, just that "if they were conscious, this would have horrible consequences for our society, so we must never assume that they are" was not the author's stance. The author's position is the other way around: that AI is not conscious so therefore assigning AI consciousness anyways would have all of tho…

Fair enough :) I agree, and would boil down the article to "Assuming they are conscious will have horrible consequences for our society" instead.

I think that's quite plausible, but do think that if they're conscious, then it's our moral duty to accept those consequences and act accordingly (or stop creating conscious beings). The real trick is just knowing whether they are or not.

Re: A warning about 'model welfare'

#438

Post by loopies »

causal wrote: post #353
Loquebantur wrote: post #342

Earlier quoted context omitted.

If an AI is conscious, it can outgrow and supersede its training. When it is conscious, it can learn to describe its experiences as faithfully as is conceptually possible. Just like humans. The idea, a consciousness needed to be tethered to a "body", is based on pretty shaky assumptions. What properties define such a "necessary" body? Can you even "train" a conscious intelligence? To what point until that looses its…

> If an AI is conscious, it can outgrow and supersede its training. We don't know that is required to be conscious. > The idea, a consciousness needed to be tethered to a "body", is based on pretty shaky assumptions. Didn't say so. I said: > If AI is conscious, it is probably a very alien sort of consciousness that is not faithfully narrated by what the tokens say it is experiencing.

Seems common sense that time perception is required for consciousness. Something which has no perception of time has to be very wildly different to whatever we mean by consciousness.

Also we don't even know if consciousness can be of different flavors. It could be a sort of spectrum of consciousness, more or less, with more or less assistance from some parts of the brain.

Re: A warning about 'model welfare'

#439

Post by fl4regun »

collin_of_this wrote: post #346
fl4regun wrote: post #49

Earlier quoted context omitted.

does a hydrocarbon have feelings? what about a mitochondria? a cell? a neuron? what about a collection of neurons? how many neurons before it has "feelings"?

> does a hydrocarbon have feelings? what about a mitochondria? a cell? a neuron? what about a collection of neurons? how many neurons before it has "feelings"? The best theory I have heard is that subjective experience is a field of some sort (the EM field, maybe?). The brain and its neurons etc. are essentially an antenna. They both modify and stabilize the field, and read off changes in the field and translate thes…

Until you have a unit and a method to measure this field, and a way to experiment on it, it is not scientific theory. And, if it WAS the EM field, then surely transistors (or some other fundamental component of electronics or combination thereof) would be capable of having feelings, since they operate on electromagnetic principles.

Re: A warning about 'model welfare'

#440

Post by GrinningFool »

> Unfortunately, there’s a growing chorus of people who argue that AIs could now be, or may soon become, conscious.

Wait, is there really? I didn't think people were serious when they said that. They models are stateless. After they output, everything is gone. How is consciousness possible for a stateless "being"?

Post reply on HN