Live data from Hacker News

A warning about 'model welfare'

mustafa-suleyman.ai

281–290 of 575 posts

Re: A warning about 'model welfare'

#281

Earlier quoted context omitted.

> [at the hearing regarding the civil rights of androids like Data] > Capt. Picard: Now, the decision you reach here today will determine how we will regard this... creation of our genius. It will reveal the kind of a people we are, what he is destined to be; it will reach far beyond this courtroom and this... one android. It could significantly redefine the boundaries of personal liberty and freedom - expanding them…

And 20-odd years later, a terrorist attack prompts Starfleet to outlaw and eradicate his entire species.

Every time I'm reminded NuTrek exists I get sad about what we've lost.

Re: A warning about 'model welfare'

#282

Earlier quoted context omitted.

Well there are, today, several models (either text or image or vid) that can be run in a fully deterministic way. A conscious machine that always answer the very exact same thing, formulated the exact same way, bit for bit, to a query is, well, quite a weird kind of "consciousness". Now, I know, I know: the counter-argument is going to be "but humans have no free-will and are 100% deterministic too" . I haven't yet d…

The idea that determinism and consciousness are incompatible is deeply intuitive to many people. And many other embrace it - you can trace this back to the debates for and against Calvinism. A lot comes down to the way people parse causation and choice. You don't want to say that a murderer was completely caused to choose something because then you can't hold the person responsible. And so determined consciousness ma…

You know, the one really interesting dynamic domain is chaotic, which is a subdomain of the deterministic! I figure free will is a chaotic phenomenon.

Groundhog day is your intuition pump here. Go back in time and most assume that the day will go mostly the same, except for the butterfly effect if you change something. Given exactly the same conditions (we went back in time, so that's pretty exact), people will make the same decisions.

Most people don't assume that the day will be completely different the next time loop. The town won't suddenly spontaneously all start breakdancing or standing on their heads.

(Edit: The murderer is the one who -given the same/similar conditions- will murder again. I'd politely suggest maybe we put them behind bars as a precaution against that; until/unless they learn to act differently. )

(Edit 2: Throw in stochasticity and this actually tends to smooth out chaotic systems. This is counter-intuitive! People's brains break on Evolution in the same way. Meanwhile somehow folks intuit birdshot without difficulty; which is weird!)

Re: A warning about 'model welfare'

#283
post #138

Earlier quoted context omitted.

Well there are, today, several models (either text or image or vid) that can be run in a fully deterministic way. A conscious machine that always answer the very exact same thing, formulated the exact same way, bit for bit, to a query is, well, quite a weird kind of "consciousness". Now, I know, I know: the counter-argument is going to be "but humans have no free-will and are 100% deterministic too" . I haven't yet d…

> A conscious machine that always answer the very exact same thing, formulated the exact same way, bit for bit, to a query is, well, quite a weird kind of "consciousness". You'll need to explain why.

it's self evident isn't it - such a consciousness has never existed, one existing would be by the fact of existing be weird

Re: A warning about 'model welfare'

#284

Earlier quoted context omitted.

>The solution isn't 'just don't use them' but rather how to use these things as ethically as possible. Bollocks. If you truly believe these LLMs are soon to have something resembling consciousness and agency, then what you're really saying is "how do we do slavery, but ethically".

Do you say the same thing for the shoes you wear, the clothing you wear, the device you are typing on right now? A chain of people were likely not paid well, some of them not at all, to produce those things. We don't ignore that it happens, but try to minimize it, knowing it's never going to be eliminated. Do you forsake all animal products, because they are conscious beings?

>Do you say the same thing for the shoes you wear, the clothing you wear, the device you are typing on right now?

No, because the shoes and clothes I wear are not living, sentient beings.

Re: A warning about 'model welfare'

#285
post #138

Earlier quoted context omitted.

Well there are, today, several models (either text or image or vid) that can be run in a fully deterministic way. A conscious machine that always answer the very exact same thing, formulated the exact same way, bit for bit, to a query is, well, quite a weird kind of "consciousness". Now, I know, I know: the counter-argument is going to be "but humans have no free-will and are 100% deterministic too" . I haven't yet d…

> A conscious machine that always answer the very exact same thing, formulated the exact same way, bit for bit, to a query is, well, quite a weird kind of "consciousness". You'll need to explain why.

Why it's weird? Seems obvious. Because it doesn't behave any differently than any other process that we don't think of as conscious. We don't think of a valve as conscious; and if we make a Rube Goldberg machine valve, we don't think of it as more conscious. It's just a series of valves that do a predictable thing.

I don't think it's defensible to say

1) that a transistor isn't conscious,

2) that a bunch of transistors that I've wired together aren't conscious, because I know how I've wired them together and therefore I know when I give them a particular input I will get a particular output, just like the single transistor, then go to

3) that I have a bunch of transistors that I've wired together, but I put in so much input that I can't remember exactly what I've put in, plus I've wired some of the transistors to output random numbers that would be difficult to guess and fed them in also, therefore I don't know what will come out, are conscious.

Even if I do accept this, if I take away the random number generator, and I can literally predict what can come out (by running a test in advance), and I still considered that consciousness, that would be odd. The only reason why I ever suspected consciousness was because I couldn't predict the output. I shouldn't even have accepted that, because I didn't think of the random number generator as conscious. [edit: of course, with the same seed the random number generator would have the same oddness.]

It seems a bit like an argument from ignorance, a theological argument. Not understanding how something moves makes it alive (animated by spirits.) But it's even weirder to assign it metaphysical qualities when you built every single element of it with the goal to do the thing that it does, and it does it totally predictably and deterministically.

Re: A warning about 'model welfare'

#286
post #102

Earlier quoted context omitted.

This is a strawman. Suleyman is not arguing that the human brain cannot be replicated. LLMs are nothing like human brains. Cargo-culting consciousness is not replicating the brain, not even a tiny bit.

Cephalopod and decapod brains aren't much like human brains either, yet it's still reasonable to regard them as being sentient and worthy of protection from unnecessary suffering.

They have a lot more in common with humans than LLMs have with either. Complexity is not evidence of consciousness.

Re: A warning about 'model welfare'

#287
> AIs are not conscious. They do not feel, experience, or suffer. They do not have innate preferences or underlying motivations.

That should be pretty obvious to anyone who ever created a chatbot using the top LLM APIs:

You can send the same question 1 million times to the same API, and it won't get tired from answering it. But if you simulate a conversation where the same question is repeated 10 times, it will auto-complete the text in a way that seems human. However: you can manipulate it by changing the conversation history; you can reset, roll back and branch the conversation at any point.

Re: A warning about 'model welfare'

#288
post #116
post #102

Earlier quoted context omitted.

This is a strawman. Suleyman is not arguing that the human brain cannot be replicated. LLMs are nothing like human brains. Cargo-culting consciousness is not replicating the brain, not even a tiny bit.

I'd argue back that using the substrate as a reason why there should not be consciousness seems quite weak. A competing thesis is that what matters is the emergent properties, whatever the support is. So far it has been true for some really high-level tasks - writing coherent text, programing, following instructions, analyzing images... I do not see why the substrate argument would work specifically for consciousness…

My response was to a comment appealing to brain reconstruction. My point is that a brain may be reconstructed, but LLMs do nothing of the sort. Substrate was never the point.

Re: A warning about 'model welfare'

#289
post #138

Earlier quoted context omitted.

> A conscious machine that always answer the very exact same thing, formulated the exact same way, bit for bit, to a query is, well, quite a weird kind of "consciousness". You'll need to explain why.

it's self evident isn't it - such a consciousness has never existed, one existing would be by the fact of existing be weird

It doesn't seem self evident to me, no.

I suspect the reasoning is connected to free will vs. determinism. But no, I see no inconsistency. You'll have to actually point it out.

Re: A warning about 'model welfare'

#290

> AIs are not conscious. They do not feel, experience, or suffer. They do not have innate preferences or underlying motivations. That should be pretty obvious to anyone who ever created a chatbot using the top LLM APIs: You can send the same question 1 million times to the same API, and it won't get tired from answering it. But if you simulate a conversation where the same question is repeated 10 times, it will auto-…

This has nothing to do with the statement above- it's just a consequence of the fact that by resetting the context you reset all memory of the llm. The same would happen if you could reset entirely a human brain to some initial state- and it would mean nothing regarding its consciousness or ability to feel.
Post reply on HN