Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A warning about 'model welfare'

mustafa-suleyman.ai

Post reply on HN
Print view View original

661–670 of 676 posts

Re: A warning about 'model welfare'

#661

Post by famouswaffles »

HarHarVeryFunny wrote: post #656
famouswaffles wrote: post #652

Earlier quoted context omitted.

I mean sure, but I'm not sure what that has to do with the broader point. It will learn to play, and it will have a model of what it means to win.

> I'm not sure you quite understand the full meaning of this statement. If you did, your following paragraphs wouldn't follow I was just explaining how this comment you made is wrong.

It's not wrong. You admit that LLMs will 'learn anything that helps them predict' and fail to realize the breadth of that statement. Your chess statements don't really help your case. It doesn't matter that it usually doesn't primarily care about winning. It still learnt how to play the game, and it still knows how to win. Similar outcomes for predicting emotions would mean it still developed an affective state, and that its ability to 'feel angry' is no less real.

Re: A warning about 'model welfare'

#662

Post by causal »

alch- wrote: post #473
causal wrote: post #324

Earlier quoted context omitted.

I am confused what people even think AI is experiencing. If it claims to be a bat and emits designated echolocation tokens, is it experiencing true bat-like sensations? In humans, at least, we can tie language back to shared whole-body physiological responses. The tokens from an LLM do not represent anything of the sort. If AI is conscious, it is probably a very alien sort of consciousness that is not faithfully narr…

> It can be trained to say it feels like a bat and insist upon it vigorously. So can you.

Exactly the point!

Re: A warning about 'model welfare'

#663

Post by mwigdahl »

thaneross wrote: post #281
chuckadams wrote: post #133

Earlier quoted context omitted.

And 20-odd years later, a terrorist attack prompts Starfleet to outlaw and eradicate his entire species.

Every time I'm reminded NuTrek exists I get sad about what we've lost.

Same here, then I watch _The Orville_ and I'm happy again.

Re: A warning about 'model welfare'

#664

Post by CPLX »

pixl97 wrote: post #560
CPLX wrote: post #556

Earlier quoted context omitted.

> The thing about sovereign AI What the fuck is that? It’s computer software. The variant of software that tries to harm other computers is called malware and the variant that replicates itself, spirals out of control, and spawns from other people's computers is called a computer virus. We've seen this kind of thing before. Yes this will be different, just like the Morris worm was different from the stuff that came b…

>Are you old enough to remember the Y2K hysteria? I'm old enough to have actually fixed Y2K problems so your world kept working the next day. If everyone ignored it the first would have been a very messy day (well generally long before that with financial systems). Y2K wasn't an issue because we worked to fix it. >our life will completely and totally change for the past 30 years or so I mean when I was a kid there wa…

The word sovereign has a specific meaning beyond simple independence. It means being in charge and above all laws.

There’s no such thing as “sovereign AI” because AI is a collection of algorithms. It’s entirely composed of on/off states in a collection of transistors.

Re: A warning about 'model welfare'

#665

Post by altruios »

pamcake wrote: post #557
altruios wrote: post #424

assertion: consciousness == processing information. This is a functional view of consciousness. Awareness - sentience - follows when the system of processing information is itself part of the information being processed. Stochastic thoughts relating to this conclusion: Life is a 'process', it doesn't have a 'physical representation'. Life is generated entirely from non-living material. "oh my cells are alive", but th…

With that assertion, every running PC has consciousness and a self-bootstrapping compiler or a debugger attached to itself are sentient? Doesn't seem meaningful.

> debugger attached to itself are sentient?

No standard debugger is capable of being attached to it's own instance.

Nor does a self-bootstrapping compiler point towards it's own instance.

I think from these two examples I should clarify:

A process that processes itself as an input, is definitionally self-aware.

Re: A warning about 'model welfare'

#666

Post by lioeters »

mitxela wrote: post #595
lioeters wrote: post #499

Earlier quoted context omitted.

As long as the machine is part of the larger universe, and not completely isolated in its own bubble of space-time (an impossible situation, an object without an environment) - it cannot be purely deterministic because of the fundamental nature of physics.

So your theory is that arithmetic is nondeterministic because very rarely a calculator will say 1+1=3.

Arithmetic is deterministic, because it is abstract. A calculator or computer is not, because it is physical and exists in a non-deterministic universe. But with error correction, etc., it's deterministic enough for practical purposes. For more in-depth explanation, start with https://en.wikipedia.org/wiki/Indeterminism

Re: A warning about 'model welfare'

#667

Post by vidarh »

chrisjj wrote: post #648
vidarh wrote: post #622

Earlier quoted context omitted.

Does your disc drive have the complexity of an LLM that shows behaviour consistent with having emotions? I'd consider that a prerequisite , but not sufficient , for that question to make sense.

> Does your disc drive have the complexity of an LLM that shows behaviour consistent with having emotions? An LLM does not show behaviour consistent with having emotions. It's regurgitation of humans' written accounts of emotions is not "having emotions".

Your second sentence does not support the claim in your first sentence; it's merely a second unsupported claim. It is also entirely irrelevant to the argument.

The threshold for behaviour consistent with having emotions (note: not behaviour sufficient to prove emotions) was one I purposefully set extremely low, because your comparison to a disk drive is so ludicrous, and yet your disc drive will still not clear it.

Do you have an actual argument?

Re: A warning about 'model welfare'

#668

Post by lukifer »

mitxela wrote: post #583
lukifer wrote: post #501

Earlier quoted context omitted.

I don’t have a good answer beyond common-sense intuition. It’s precisely because I don’t have a good answer that I leave open the possibility. But to extrapolate a bit, theres a sense in which it cashes out to a difference between abstracted representation and embodiment. If GFLOPS can be conscious, does that mean a billion scribes doing those operations on paper would also have consciousness, even identical consciou…

Would an atom-accurate simulation of a human being carried out by scribes be conscious? We can't say yes. And we can't say no.

Exactly.

Re: A warning about 'model welfare'

#669

Post by HarHarVeryFunny »

famouswaffles wrote: post #661
HarHarVeryFunny wrote: post #656

Earlier quoted context omitted.

> I'm not sure you quite understand the full meaning of this statement. If you did, your following paragraphs wouldn't follow I was just explaining how this comment you made is wrong.

It's not wrong. You admit that LLMs will 'learn anything that helps them predict' and fail to realize the breadth of that statement. Your chess statements don't really help your case. It doesn't matter that it usually doesn't primarily care about winning. It still learnt how to play the game, and it still knows how to win. Similar outcomes for predicting emotions would mean it still developed an affective state, and…

I said an LLM will learn anything that helps it to predict, then gave examples of playing chess by prediction and predictive emotions, both of which you seem to now accept, so you are now accepting that my "following paragraphs" did in fact follow. Go figure!

You want to argue that predictive emotions are just as real as animal emotions, but that doesn't stop them from being predictive (and that AI that smiles as it kills you still seems concerning).

¯\_(ツ)_/¯

Re: A warning about 'model welfare'

#670

Post by lukifer »

dtj1123 wrote: post #599
lukifer wrote: post #574

Earlier quoted context omitted.

If heuristics didn't pay off more often than not, we wouldn't have them. And I'd posit many scientific did in fact begin with chasing a hunch until it became a hypothesis (followed by years/decades of the real work). But most significantly, interior experience isn't empirically measurable by definition: hence it's a "hard problem". An LLM who insists "I am alive, please don't shut me off" offers no affordances to dis…

Heuristics and common sense intuition are quite obviously not the same thing. You don't have to posit that scientific breakthroughs come from hunches, you can actually cite examples from books or Wikipedia :D. There are plenty! Again, that has nothing to do with whether or not a given scientific breakthrough is based on common sense intuition. Your point about interior experience not being empirically measurable is a…

Fair enough. But not everything has to be a breakthrough: sometimes you have to solve an immediate problem (like "how much can I trust this stranger I just met").

Whether to ascribe moral standing to a probability engine running on top of GPUs is perhaps not unlike a doppelganger scenario in sci-fi (eg The Thing): you're presented with an entity whose behavior is indistinguishable from a conscious entity capable of experiencing suffering. But you also have reason to think it might be a blind autonomous imitation process, optimized around fooling you, with no more subjectivity than an RNA virus.

I lean towards benefit of the doubt, that the virtue-ethical implications of what it would mean to dismiss potential personhood are unsettling, and we should be wary of becoming that kind of moral creature (notwithstanding the further extensions of that ethic: I don't practice veganism, for instance).

But that doesn't contradict my gut-check: that if I assume AI personhood (maybe not yet, but in the near future), there's still a good chance that it's merely math and I'm being algorithmically hoodwinked, no different than evolutionary mimicry by a bug who has no idea what it's doing.

Post reply on HN