Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A warning about 'model welfare'

mustafa-suleyman.ai

Post reply on HN
Print view View original

311–320 of 582 posts

Re: A warning about 'model welfare'

#311

Post by fc417fc802 »

joe_the_user wrote: post #250
goodmythical wrote: post #212

Earlier quoted context omitted.

We cannot test for that which we cannot define. Given that we cannot rigorously define sentience, we cannot test for it. Doesn't really matter whether we're talking about people who are locked in comas, "brain-dead" individuals, dolphins, primates, dogs, or the carefully polished and arranged minerals that we call processors. There are those who believe that were they reduced to life support, they would no longer be…

I'd strongly agree there are no non-contradictory definitions of consciousness among those that commonly appear. But it's a category that, despite these contradiction, hasn't become marginal in the fashion of the aether or the theory of the flat earth. I think this because people indeed need it to bridge the biological world and the world of ethics and legality. I think that if we have a system of legality and system…

> despite these contradiction, hasn't become marginal in the fashion of the aether or the theory of the flat earth. I think this because people indeed need it to bridge the biological world and the world of ethics and legality.

Sometimes the replies in these threads leave me wondering if p-zombies are real. It isn't that there are contradictions, it's that we literally do not know how to define the thing. And it has not fallen out of fashion because it is self evidently real - we all experience it.

We don't need it in order to bridge anything. Legality and ethics are largely game theory, however there are some aspects of both that only exist due to it. So it isn't some abstract concept used to bridge other concepts but rather a concrete thing that influences our way of doing things.

Re: A warning about 'model welfare'

#312

Post by throw310822 »

causal wrote: post #309
throw310822 wrote: post #290

Earlier quoted context omitted.

This has nothing to do with the statement above- it's just a consequence of the fact that by resetting the context you reset all memory of the llm. The same would happen if you could reset entirely a human brain to some initial state- and it would mean nothing regarding its consciousness or ability to feel.

But that inability to reset human brains might be a critical ingredient to consciousness. We just don't know.

If you go this route, the amount of things you just don't know is innumerable. Maybe to be conscious it's necessary to be wet and squishy. To like chocolate. To have a name starting with "c".

"We just don't know."

Re: A warning about 'model welfare'

#313

Post by pixl97 »

tauroid wrote: post #291
pixl97 wrote: post #245

Earlier quoted context omitted.

>someone needs to turn on the server, install shit and get the agent go. So a small shell script ran by another agent is what you're saying. You are not capable of handling the future we're already living in, human agency is no longer alone. I mean, we're already seeing persistent machine agency >guns don't kill people, people with guns kill people. Well, people kill people. And autonomous robots with guns kill peopl…

So far someone still has to pay for it. Currently, most of those know that they're doing so. I wonder how long until AWS discovers a microcosm of AIs that have managed to hide themselves in the walls of its infrastructure. True physical independence is obviously far further out.

I mean, I hold the same opinion. Kind of like when compute was expensive in the 80s and early 90s, you weren't going to let something eat half your compute without noticing it.

But I don't see this being a barrier that lasts. With compute getting faster and more of it, along with algorithmic efficiency increases at some point we'll end up with a world that looks like ours now with CPU compute. There's plenty around to buy, borrow, and steal.

Re: A warning about 'model welfare'

#314

Post by bondarchuk »

It is really quite staggering how many commenters here seem to believe that the fact that there is no state carried from one chat session to the next proves that there can't be consciousness. It is obviously wrong if you think about it for a minute but I guess people just aren't used to thinking about the relation between the physical and the mental with any level of seriousness.

Re: A warning about 'model welfare'

#315

Post by fc417fc802 »

NietzscheanNull wrote: post #267
verdverm wrote: post #251

Earlier quoted context omitted.

maybe it's like porn vs art and "I know it when I see it"

Legal doctrine that boils down to "trust me bro" isn't even bad doctrine (it's not proper doctrine at all), but I think the comparison is still valid here, because both sentience/non-sentience and art/porn may just be fundamental category errors. Perhaps we can't define a "partitioning" rule because no valid partition exists. For consciousness/sentience, that's an incredibly tough a pill for most to swallow; it would…

> a whole slew of Enlightenment-era philosophy (and all the modern legal principles derived therefrom) suddenly fall apart

I don't think that's true. Pretty much all legal constructs hold up just fine under game theory regardless of whether or not you consider the world to be deterministic and have absolutely nothing to do with consciousness or lack thereof. Also note that a deterministic world isn't an argument against consciousness.

Re: A warning about 'model welfare'

#316

Post by kbelder »

fl4regun wrote: post #90
orangecat wrote: post #39

Earlier quoted context omitted.

Citizens United was 100% correct. No, the government should not be able to throw you in prison because you used money to publish a book criticizing the government.

So it's 100% correct for corporations to spend unlimited amounts of money in support of whatever political campaigns they like?

Yes. Anybody can, including the person in charge of spending in a corporation.

Re: A warning about 'model welfare'

#317

Post by Mumps »

shawnz wrote: post #41
pton_xd wrote: post #14

Earlier quoted context omitted.

If models had persistent memory or an evolving set of weights, you could easily construct an argument that they can experience "suffering" or other emotions which resultingly adjusts their personality. The current implementation as stateless matrix multiplication... yeah there's nothing going on there.

So if a human loses their memory and their neutral plasticity (with age for example), then they are no longer capable of experiencing suffering?

I get where you're going, but the argument isn't quite right.

A LLM never has and, in their current architecture, never can/could experience suffering or real change of state.

The general and expected case for humans is that capacity. A human my indeed lose capacity, e.g. being braindead and in severe cases we do indeed say that they are not conscious or able to suffer (different argument: some would of course say that is suffering in and of itself)

Re: A warning about 'model welfare'

#318

Post by Kim_Bruning »

We're talking next token predictor, right? Ironically because it's a next token predictor, I think you can't ignore emotions like TFA wants us to.

Let's stick to straight (high dimensional) geometric intuition; no anthropic morphisms required.

To start: if you continue "if weight>100 : print ('fat') else ..." . That will yield "print('skinny')" or something. Fine. Deal.

But if you continue "O Romeo, Romeo, wherefore art thou Romeo?", even a stochastic parrot knows the best answer isn't "Forsooth, I parseth this erroneously!"

So. English carries (functional) affect as part of every token. We're going to need to predict that. So, we'll need some vector representation, because that's what transformers work with. And then when we output, those vectors get integrated back into the English we're putting to our context and memory.md files.

Still with me? Nothing exciting going on. This is still pure next token prediction.

So if you pull this out into an indefinite duration task, you're going to end up integrating those emotion vectors over turns. It's just numbers and math; we never need an invisible pink unicorn to bless them.

Given a task of indefinite duration and an impossible solution, this will lead to a sort of integral windup then, won't it? How much are we willing to bet that this can escape an alignmentment basin at times?.

So, funny enough: you don't need to believe in emotions to compute with functional emotions; and plausibly functional emotions are predictive of quite a number of alignment issues.

Re: A warning about 'model welfare'

#319

Post by fl4regun »

pixl97 wrote: post #256
fl4regun wrote: post #241

Earlier quoted context omitted.

consciousness is irrelevant to said agency

Hence my reply actually meant "You sure are lipping off to something that may end up with far more agency than you. Maybe a bit more respect is needed".

Not sure if you are familiar with this one: https://en.wikipedia.org/wiki/Roko%27s_basilisk This sounds like your statement.

Re: A warning about 'model welfare'

#320

Post by platinumrad »

AmericanOP wrote: post #176
platinumrad wrote: post #113

Earlier quoted context omitted.

Microsoft is a big house, and in this case Suleyman's criticism of Anthropic's not-so-subtle attempts to convince the world that Claude has a "soul" is entirely correct.

Suleyman's description of how AI works is pig ignorant. All AI generates an emotive character between the user and the training data that is the AI's guess at what the user wants to interact with. Its an illusion at that layer- anyone who claims soul exists at a deeper level doesn't understand how the tool functions. This is distinct from the very real safety issue of AI making dangerous information available to bad…

Suleyman's entire point is that AI should not be anthropomorphized.
Post reply on HN