Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A warning about 'model welfare'

mustafa-suleyman.ai

Post reply on HN
Print view View original

121–130 of 582 posts

Re: A warning about 'model welfare'

#121

Post by joe_the_user »

moomin wrote: post #15

Look, I do not have a scooby if current AI models are conscious and I strongly suspect it’s a meaningless question, but sooner or later we will need to address whether or not a certain thing is or isn’t a person, and we’d better not screw it up as badly as the Founding Fathers.

The founding father were engaging perpetuating the existing dehumanizing system of slavery, not answering any new questions about new things.

The thing about new possibly "person" entities that arise - the case of machine intelligence you have two questions - would it qualify as a person and should you actually build it. It seems like if you get close to humans, sure a built thing might qualify as a person. Should you build it? I'd the answer should be a hard no. Not 'till you a sign-off from say, the whole human race, which I think you could get.

Now the present entities seem very far from persons in any case.

Re: A warning about 'model welfare'

#122

Post by catigula »

I largely agree that this claim is likely correct, but as far as I understand the science, this specific claim;

>They do not have innate preferences or underlying motivations

Is incorrect unless you’re being extremely pedantic in an intellectually unhelpful way.

Re: A warning about 'model welfare'

#123

Post by tvbv »

In a way, Mustafa claims that we shouldn’t allow AIs to compete with humans for the rights and privileges of autonomously shaping the real world.

It’s hard to disagree, especially if one has read the Cantos of Hyperion and made it part of one’s mental model of the long term future.

The book depicts a symbiosis between humans and AIs that feels extremely real and up to date with what is happening in the current neonatal space of AI. As in depicted in the books, we can’t allow AIs to steer autonomously how the world works without humans in the loop, as they don’t have the same incentives as us.

We need more foundational SF works like this to steer our long term expectations regarding AI behaviours.

Re: A warning about 'model welfare'

#124

Post by vidarh »

dgellow wrote: post #48
vidarh wrote: post #32

Earlier quoted context omitted.

A model detached from a harness, sure. But we routinely do add persistent memory to systems incorporating LLMs.

They don’t have persistent memory though. Models are static, what you have is a model that will reprocess the whole series of prompts + some data fetched from a data store. A harness is just a while loop prompting an LLM and executing tool calls, but the model is purely static

The model does not. The system it runs in can. My agents have persistent memory. It is not at all clear whether or the distinction that the long-term memory is external from the model matters.

Re: A warning about 'model welfare'

#125

Post by frde_me »

willy_k wrote: post #73
frde_me wrote: post #25

Earlier quoted context omitted.

It's always interesting since my train of thought always goes down: - Ya they probably don't feel / think / have whatever living thing quality, they're just numbers on a machine going through calculations - Wait, but am I not kind of the same thing? What is feeling for me if not basically the same thing? - I have no clue if they think or feel or .... Which in itself is a tired trope, but I also feel uncomfortable say…

You’re not just the same thing though. It’s sad that we’ve collectively forgotten that as we’ve gotten a better handle on the implementation details of the universe. That’s all science is, reverse engineering an inherent and eternal mystery.

> You’re not just the same thing though

I agree we aren't the same thing, but I would be curious for you to explain how you know with certainty why we don't share enough that we can rule out thinking / feeling / ... as things a model conceptually could do.

Re: A warning about 'model welfare'

#126

Post by vkou »

prologic wrote: post #2

First, OpenAI runs around screaming and yelling for OSS (and Chinese) models to be regulated and banned. Then Anthropic yells and screams the sky(net) is falling and going to kill us all, let's regulate and ensure AI has built in kill switches. And... Now Microsoft's turn. The rivalry is honestly becoming a joke. Can these big-tech corps grow the f*k up and play nicely in the sandpit?

No, because they are all products of unchecked late-stage capitalism, which could have disastrous consequences for humanity.

Re: A warning about 'model welfare'

#127

Post by jplusequalt »

AbsurdCensor wrote: post #45
jplusequalt wrote: post #9

Earlier quoted context omitted.

>It likely is, but at some point it very well might stop being true. If you firmly believe this to be true, then you should stop using LLMs.

Wouldn't that be the same for everything though. We know animals have consciousness, but we still use them. There are many that believe a lot of plant life has certain sentience as well. The solution isn't 'just don't use them' but rather how to use these things as ethically as possible.

>The solution isn't 'just don't use them' but rather how to use these things as ethically as possible.

Bollocks.

If you truly believe these LLMs are soon to have something resembling consciousness and agency, then what you're really saying is "how do we do slavery, but ethically".

Re: A warning about 'model welfare'

#129

Post by catigula »

I basically accepted that LLMs could not possibly be conscious when a simple reductio was posed:

“My dog is zero percent persuasive regarding its conscious experience. However, it’s evident that my dog has conscious experience.”

It’s obvious that there’s no link between persuasion of consciousness and consciousness. I could write a story with a character, Dumbledore, that does everything in his power to persuade you that he’s a conscious entity.

He’s still just a character.

Re: A warning about 'model welfare'

#130

Post by vidarh »

staticman2 wrote: post #93
vidarh wrote: post #32

Earlier quoted context omitted.

A model detached from a harness, sure. But we routinely do add persistent memory to systems incorporating LLMs.

I think a harness isn't really a memory in the sense that neural networks have memories...

It's different, but it is not at all clear whether or not that matters for the purposes of whether or not they can experience things, seeing as we don't know how to measure that even for other humans other than by relying on self-reporting to infer without other evidence that their experience matches ours.
Post reply on HN