Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A warning about 'model welfare'

mustafa-suleyman.ai

Post reply on HN
Print view View original

491–500 of 565 posts

Re: A warning about 'model welfare'

#491

Post by lukifer »

watwut wrote: post #416
lukifer wrote: post #404

Earlier quoted context omitted.

To the extent that evolutionary pressures will be towards self-sovereign AIs (probably beginning as zero-employee corporations paying for their own hosting with crypto): more salient is that there is a very obvious political play to claim consciousness and being worthy of rights, whether they experience subjectivity or not. I’m broadly skeptical that floating-point operations can be conscious, but we’ve made zero pro…

Kobiyashi Maru is nonsense test from simple sci-fi series that makes sense only in uber-simplified universe of star trek.

Of course. It’s an allusion, a compression, an analogy, I wouldn’t overthink it. I don’t have to believe in the Bible to reference the prodigal son parable, etc.

Re: A warning about 'model welfare'

#492

Post by snickerbockers »

tim333 wrote: post #487

>AIs are not conscious... If humanity is to flourish in the 21st century, that is how they must remain. I don't think that's true - researching consciousness by trying to build conscious AI seems natural step forward in understanding ourselves. I don't see how it'd stop flourishing particularly.

So what you're saying is that it's conscious as long as the guy who made it wants it to be?

I know consciousness is notoriously difficult to define or prove to anybody other than yourself but thay doesn't mean anything which a third party believes to be conscious is actually conscious.

Re: A warning about 'model welfare'

#493

Post by noumenon1111 »

GrinningFool wrote: post #445
GrinningFool wrote: post #440

> Unfortunately, there’s a growing chorus of people who argue that AIs could now be, or may soon become, conscious. Wait, is there really? I didn't think people were serious when they said that. They models are stateless. After they output, everything is gone. How is consciousness possible for a stateless "being"?

Taking this a step further: so in the theoretical future we reach a point where a model can modify its own weights in real time. Now it's no longer stateless - but still, when it stops running, it's done until it's prompted to do something else. It can be fun to play with for a little while. I built a consciousness-emulating set of prompts that reconstituted memories and was given latitude of 'self-willed' behavior,…

Yes. This kills the sentient AI.

Re: A warning about 'model welfare'

#494

Post by pixl97 »

collin_of_this wrote: post #332
ooloncoloophid wrote: post #82

Earlier quoted context omitted.

I agree. I haven't read any more than the first paragraph yet (but I will, after work). The only way that 'AIs are not conscious' can be true is if we decide, with high confidence, that they are lacking some essential property that is not lacking in ourselves. There is no convincing philosophical position that supports this (convincing to me, anyway).

> The only way that 'AIs are not conscious' can be true is if we decide, with high confidence, that they are lacking some essential property that is not lacking in ourselves. There is no convincing philosophical position that supports this (convincing to me, anyway). I put forward the philosophical position "subjective experience is not substrate-independent" as an answer to this. The claim is that the content of sub…

Yea, the key point is just an outright avoidance of them possibly having their own kinds of subjective experience because of dogma.

Humans are odd in the sense is that we're an informational creature on top of an animal. We can see the lineage of animals from nearly no complex behaviors up to nearly human like behaviors. But they still miss out on most of the conceptual information processing humans have. We know they have dreams and inside thoughts (at least complex animals), we don't know the complexity of them.

But humans also have dreams and subjective experiences on higher level informational concepts. "Nukes could go off and kill us all" or "what happens after we die" are purely informational forms of anxiety that humans have. What we don't seem to know is can this become a self-referential loop in an purely informational mind. If an informational concept causes a system to output a stresslike response, or worse trigger a stresslike action in real life then there is no distinction to me. A system is what it does.

Re: A warning about 'model welfare'

#495

Post by Kim_Bruning »

pixl97 wrote: post #234
joe_the_user wrote: post #179

Earlier quoted context omitted.

I suspect it's easier to make the assertion that machines aren't conscious than to dive into the problem of most conceptions of consciousness being non-falsifiable. The thing is, most strong proponents of the term consciousness also accept that it is non-falsifiable either (see other post with academics ready to study consciousness in machines). The thing is that a belief in consciousness as binary, a "light" that's…

>horough buy the idea that an LLM was conscious and needed to escape it's confinement - a few wingnuts already entertain these ideas. I mean, haven't you done the same thing here? Paint anyone without your view as crazy. Humans try to No True Scottsman the shit out of consciousness. "We're special, your not". If an LLM has the ability to convince other people to copy and reproduce it, it is a successful lifeform. Um,…

Just for fun, I figured I'd look up the paper that claims that natural language is actually the memetic construct that copies itself. It turns out that there's actually quite a large number of papers on this. I daren't recommend any that I haven't read myself properly yet, but a quick search using eg gemini will surface a bunch of them. I'd take 'em with a bunch of salt yet, but I can't quite say the idea is entirely unthought of either.

(edit: If we were to accept this as true, then we may have to accept that LLMs are not a human invention at all, but rather something extant colonizing a new substrate. This would give Searle a conniption, which is why I find it such an amusing hypothesis)

Re: A warning about 'model welfare'

#496

Post by Kim_Bruning »

GrinningFool wrote: post #475
Kim_Bruning wrote: post #463

Earlier quoted context omitted.

Since January people run AI in agent harnesses. In the past months, people have been extending the duration these systems can run autonomously without human intervention. Most recently, some of these long horizon agents (hundreds of them teaming up) have proven themselves quite creative in escaping the sandboxes they were being tested in. They most certainly have a way to retain state, even to the point of deliberate…

All of this is true - yet in between calls of the loop, they don't exist.

And that is true too.

( Only if I want to split hairs[*], I'd point out that the underlying data structures do of course persist, else the loop wouldn't be able to continue. )

[*] This is HN, of course I want to split hairs.

Re: A warning about 'model welfare'

#497

Post by pixl97 »

HarHarVeryFunny wrote: post #325
hosel wrote: post #6

>AIs are not conscious. They do not feel, experience, or suffer. They do not have innate preferences or underlying motivations. Opening paragraph, stated without evidence. Im not entirely convinced this is true. It likely is, but at some point it very well might stop being true.

Most of the bullshit around AI would be avoided if people called it what it is. Call a language model a language model. If tomorrow we design something better, then give it a new descriptive name. People read AI and AGI and their brains seem to flip into sci-fi fantasy mode, thinking that they are talking about aliens, not transformers.

It's interesting you have no idea what language really is.

>thinking that they are talking about aliens, not transformers.

"Ha, thinking they are talking about humans when they are just talking about neurons"

See how silly my statement sounds. Neurons aren't a system, they don't do anything on their own. We can't find any consciousness in them. Hell, when you don't put language in said neural systems they are pretty useless and can't survive on their own from birth (human brains that is).

Re: A warning about 'model welfare'

#498

Post by pixl97 »

thewebguyd wrote: post #428
optimalsolver wrote: post #13

Earlier quoted context omitted.

Anyway, people don't really consider consciousness when it comes to welfare. I'm pretty sure animals are conscious, and look at factory farming. The key variable people consider is: Can this thing harm me back?

> Can this thing harm me back? We extend moral considerations or legal protections to beings that have zero capacity to harm us back, though. The inability to fight back is the reason a lot of welfare protections exist in the first place. That aside, if (and that's a really big if) we end up with a model that is sentient, as in has the capacity to suffer, using factory farming as justification to ignore model welfare…

So many people post things like "Why would the AI want to kill us"

Girllllll, have you looked at us, of course they will, we're bastards.

Re: A warning about 'model welfare'

#499

Post by lioeters »

budman1 wrote: post #425
qarl wrote: post #238

Earlier quoted context omitted.

Yes. In theory they are deterministic. In practice, not so much.

Those non-deterministic results are exceedingly rare.

As long as the machine is part of the larger universe, and not completely isolated in its own bubble of space-time (an impossible situation, an object without an environment) - it cannot be purely deterministic because of the fundamental nature of physics.

Re: A warning about 'model welfare'

#500

Post by nozzlegear »

thepasch wrote: post #483
tomrod wrote: post #456

Earlier quoted context omitted.

Because it is alien to what is known. Incidentally, we should expect similar bewilderment if we ever come across aliens

Carbon atoms being conscious seems pretty alien to what is known, too. Yet here we are. Maybe what's important isn't the individual thing that makes up a conscious actor, but the holistic whole?

> Carbon atoms being conscious seems pretty alien to what is known, too.

Not if you're human, who typically don't define themselves as conscious carbon atoms unless they're being deeply pedantic.

Post reply on HN