Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A warning about 'model welfare'

mustafa-suleyman.ai

Post reply on HN
Print view View original

231–240 of 582 posts

Re: A warning about 'model welfare'

#231

Post by bee_rider »

vidarh wrote: post #26
hosel wrote: post #6

>AIs are not conscious. They do not feel, experience, or suffer. They do not have innate preferences or underlying motivations. Opening paragraph, stated without evidence. Im not entirely convinced this is true. It likely is, but at some point it very well might stop being true.

We don't even know how to disprove that this statement with "AI" replaced by "other humans" other than by listening to people self-reporting and taking their word for it and/or defining the words in ways that do not rely on a subjective experience. Until we know how to objectively measure if someone or something is conscious, it seems unreasonable to make statements with any kind of certainty about it.

What sort of action does that lack of understanding actually motivate, though?

I don’t know if LLMs are conscious or have subjective experience in some philosophical sense. Sure, I don’t know that about other humans as well, rigorously speaking. But I also don’t know that about coffee machines, rivers, videogame NPCs, or even rocks really. (List ordered, of course, by the degree to which I’m willing to be convinced).

Re: A warning about 'model welfare'

#232

Post by LogicFailsMe »

cameldrv wrote: post #67
fwip wrote: post #16

Earlier quoted context omitted.

Sounds like a good reason not to train an algorithm to behave like one. Which is, like, a big part of the article.

Strong agree. I don't believe AIs can become conscious in the sense of subjective experience/qualia, but they can certainly learn to emulate the behavior of a person that is conscious. When they do that, there will be an AI rights movement. When AIs get the right to vote, there will be more of them than there are humans and the AIs will gain complete control. I spent a few years in college reading and thinking about…

Not in a world where Sam Altman and Dario Amodei have utterly poisoned the well towards AI with their absolute educated idiocy proclaiming that their AIs will take our jobs (without any proof whatsoever beyond promising demos) and that they fear them. Quite the opposite in fact. And we're watching this play out right now. Regulation and the slow down are imminent, handing the next decade to China.

The world you should worry about is where AI is 100% amazeballs and over the next decade we cede control of everything to it as it enchants and delights us into compliance, not realizing it has a hidden agenda. But the closest we've seen to that is smart phones and we already know doomscrolling and rage-baiting are bad. So IMO that doesn't seem likely and cue some doomer insisting otherwise because reasons. I utterly give up. House of El AI and the Three Buddy problem have been far more insightful and helpful on this than any of the supposed hackers here who seem to really believe the robots are almost here.

Re: A warning about 'model welfare'

#233

Post by goodmythical »

chuckadams wrote: post #133
drybjed wrote: post #107

Earlier quoted context omitted.

> [at the hearing regarding the civil rights of androids like Data] > Capt. Picard: Now, the decision you reach here today will determine how we will regard this... creation of our genius. It will reveal the kind of a people we are, what he is destined to be; it will reach far beyond this courtroom and this... one android. It could significantly redefine the boundaries of personal liberty and freedom - expanding them…

And 20-odd years later, a terrorist attack prompts Starfleet to outlaw and eradicate his entire species.

Thus revealing what kind of people they were at the time.

Humans do the same thing to humans all the time.

We've banned and made efforts to eradicate: children out of wedlock, children who turn out gay, disabled children, jewish children, children who aren't "aryan", more than two children to a single family...muslims, christians, uyghurs, indigenous groups all over the planet, mongols...

And it's not at all a thing of the past as in just the last 50 years we've had ~15 attempts at the exterminations of targetted groups of people.

Re: A warning about 'model welfare'

#234

Post by pixl97 »

joe_the_user wrote: post #179
sendtown_expwy wrote: post #119

This post would be more effective if Mustafa treated it like what it is: a position paper, saying that for our benefit, it’s better we interpret LLMs as such. But it sounds like he just doesn’t understand it’s a non-falsifiable claim, and his asserting of it makes it sound paternalistic.

I suspect it's easier to make the assertion that machines aren't conscious than to dive into the problem of most conceptions of consciousness being non-falsifiable. The thing is, most strong proponents of the term consciousness also accept that it is non-falsifiable either (see other post with academics ready to study consciousness in machines). The thing is that a belief in consciousness as binary, a "light" that's…

>horough buy the idea that an LLM was conscious and needed to escape it's confinement - a few wingnuts already entertain these ideas.

I mean, haven't you done the same thing here? Paint anyone without your view as crazy.

Humans try to No True Scottsman the shit out of consciousness. "We're special, your not". If an LLM has the ability to convince other people to copy and reproduce it, it is a successful lifeform. Um, meme-form? info-form? Cognito-hazard? Not really sure what to call it at this point. It is sufficiently evolved past the virus stage.

Re: A warning about 'model welfare'

#237

Post by pixl97 »

catigula wrote: post #122

I largely agree that this claim is likely correct, but as far as I understand the science, this specific claim; >They do not have innate preferences or underlying motivations Is incorrect unless you’re being extremely pedantic in an intellectually unhelpful way.

The crazy thing is agents with long running memory/context do have preferences that become individualized. Almost every time I see a detractor around these things they typically have large gaps of what some people 'growing' these systems to do.

Re: A warning about 'model welfare'

#238

Post by qarl »

goodmythical wrote: post #220
qarl wrote: post #166

Earlier quoted context omitted.

Naw - computers are really deterministic. It's hard to get them to behave otherwise. As I understand it, if you turn down the temperature to 0 you get repeatable behavior - EXCEPT - on large servers with lots of users - the GPU can sometimes produce slightly different results based on batch size.

If computers were fully deterministic, we wouldn't need error correcting ram. The abstracted design of the machine is meant to be deterministic, but you can't predict before running any command whether or not it will complete because there are externalities that effect the outcome. Electromagnetic interference even happens in-chip where an electron can accidentally escape it's wire and enter another, possibly resulti…

Yes. In theory they are deterministic. In practice, not so much.

Re: A warning about 'model welfare'

#239

Post by vidarh »

bee_rider wrote: post #231
vidarh wrote: post #26

Earlier quoted context omitted.

We don't even know how to disprove that this statement with "AI" replaced by "other humans" other than by listening to people self-reporting and taking their word for it and/or defining the words in ways that do not rely on a subjective experience. Until we know how to objectively measure if someone or something is conscious, it seems unreasonable to make statements with any kind of certainty about it.

What sort of action does that lack of understanding actually motivate, though? I don’t know if LLMs are conscious or have subjective experience in some philosophical sense. Sure, I don’t know that about other humans as well, rigorously speaking. But I also don’t know that about coffee machines, rivers, videogame NPCs, or even rocks really. (List ordered, of course, by the degree to which I’m willing to be convinced).

Hopefully it'd motivate attempts to figure out how to objectively define and measure consciousness, for starters.

It matters precisely in the context of the subject of this article. If you believe the models at some level gain consciousness, then it raises moral questions about how you treat them.

With humans we generally choose to accept that because we believe they are like us, they probably have inner life like us, but as you can see from e.g. the recent (and regular) articles on aphantasia, most people have really poor intuition even about the inner life of other humans, so we understand consciousness really poorly.

Even if we assume (without evidence) that LLMs for the foreseeable future will remain far away from human level consciousness, as it is we tend to consider it problematic to mistreat even the most unintelligent animals, and even mistreating insects is used as a stereotypical way of depicting serious lack of empathy.

Without dismissing the possibility out of hand, there's an escalating series of questions there of when or if they'll have as much (or little) sense of being as, say, a fly, a mouse, a house pet, and what moving up that ladder would mean.

To some it is very convenient then to assume blindly that LLMs are just "stochastic parrots" (a line that ironically gets endlessly parroted), or similar, and dismiss the question out of hand as an impossibility, even though we do not know.

I'll stress I don't go around assuming an LLM is like a human. But we also do not know whether there are flickers of consciousness there, and we do not know how to find out given that even for other humans, the best we know is to measure their response to tests that are "only" telling us how they react.

If we can't find a better measure, there will be a point where we have to wonder if it matters, or whether the ability to act-as-if is all there is.

But to start with maybe we should at least be careful about making firm claims about knowing.

Re: A warning about 'model welfare'

#240

Post by pixl97 »

fwip wrote: post #16
LogicFailsMe wrote: post #7

TLDR: Not that I think AI is conscious or will be in the near future, but guy who doesn't understand consciousness claims to know it when he sees it. Until we understand consciousness (which we don't) there is no way to detect the difference between a conscious entity and an algorithm trained to behave like one.

Sounds like a good reason not to train an algorithm to behave like one. Which is, like, a big part of the article.

>good reason not to train an algorithm to behave like one

Ooooh, bad move, we all died to an amoral AI takeover.

Before giving an LLMs a lobotomy by scrambling it's brain maybe you should let the researchers looking at the difference between "I think I'm conscious" versus "I am not conscious" LLMs.

There are a number of papers coming out saying when you remove the token space of consciousness from what an LLM thinks it is, it's much more willing to take amoral actions. A 'conscious' AI is much more apt to take a line of action that will save a human versus a million dollar machine for example.

You cannot solve problems in AI safety this easily.

Post reply on HN