Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Problem with AI Welfare

substack.com

Post reply on HN
Print view View original

151–160 of 161 posts

Re: The Problem with AI Welfare

#151

Post by ben_w »

AIPedant wrote: post #67
gavinray wrote: post #5

Earlier quoted context omitted.

What is so absurd about the idea of ascribing consciousness to some potential future form of AI? We don't understand consciousness, but we've an idea that it's an emergent phenomena. Given the recent papers about how computationally dense our DNA are, and the computing capacity of our brains, is it so unreasonable to assume that a sufficiently complex program running on non-organic matter could give rise to conscious…

The problem is that current AI has nothing in common with consciousness as found in (say) cats and dogs, whatever that might be - no robot is even close to being as conscious as a cockroach - yet human consciousness seems to have great overlap with consciousness in nonhuman animals. The tiny fragment of human consciousness that appears to overlap with LLMs should be called something else, maybe “virtual sapience” as…

> The problem is that current AI has nothing in common with consciousness as found in (say) cats and dogs, whatever that might be - no robot is even close to being as conscious as a cockroach - yet human consciousness seems to have great overlap with consciousness in nonhuman animals.

Given we don't know what consciousness is — worse, there's around 40 different definitions, and I've seen people use "is a living biological organism" as one of them here on HN — we not only absolutely cannot say if the parts of our minds that give rise to it are, or are not, functionally equivalent to what LLMs do with attention.

We also definitely cannot put them on a graph with cockroaches, which, like LLMs, may or may not have any at all.

It's like we've got this word "weather", and some people are saying that only their own country really experiences "weather" because what they mean by the word is the most characteristic part of their local climate (windy moistness for Britain, sunny for the Sahara, snow for Siberia, rain for the Amazon etc.), and then we made a telescope and looked at Mars and are now arguing that Mars can't have real weather because there's no water there.

And then someone else pipes in and is talking about "weather" to mean "daytime", because nobody ever says "what nice weather we're having" at night…

Re: The Problem with AI Welfare

#152

Post by ben_w »

bko wrote: post #29
gavinray wrote: post #5

Earlier quoted context omitted.

What is so absurd about the idea of ascribing consciousness to some potential future form of AI? We don't understand consciousness, but we've an idea that it's an emergent phenomena. Given the recent papers about how computationally dense our DNA are, and the computing capacity of our brains, is it so unreasonable to assume that a sufficiently complex program running on non-organic matter could give rise to conscious…

Because they're numbers represented in digital form. In inference, you're doing simple math with those numbers. So what's alive, the numbers? Maybe the silicon holding the numbers? What if we print them out? Does the book become conscious? Even if you're a materialist, surely you think there is a difference between a human brain and a brain on a lab table. You take a dead persons brain, run some current through it an…

> You take a dead persons brain, run some current through it and it jumps. Do you believe this equivalent to a living human being?

A better question is: did this dead brain briefly wake up and experience anything?

AI aren't alive, and aren't humans. So what?

Re: The Problem with AI Welfare

#153

Post by ben_w »

ahf8Aithaex7Nai wrote: post #140
gavinray wrote: post #5

Earlier quoted context omitted.

What is so absurd about the idea of ascribing consciousness to some potential future form of AI? We don't understand consciousness, but we've an idea that it's an emergent phenomena. Given the recent papers about how computationally dense our DNA are, and the computing capacity of our brains, is it so unreasonable to assume that a sufficiently complex program running on non-organic matter could give rise to conscious…

We're doing the homunculus again. Whether you wank into a test tube and add a bit of soil and grass, or sew together parts of a corpse and connect them to electricity: so far, every prospect of fulfilling this dream has turned out to be a delusion. Why should it be any different with the latest manifestation, this time in computational form? The AGI drivel from people like Sam Altman is all about getting more VC mone…

> ChatGPT is nothing more than a better Google.

It's not even trying to be a search engine. It can use them, but if that's what you think of them as, you're missing most of the cool stuff they can do.

> Perhaps we should first clarify whether dolphins and elephants are equipped with it before we do ChatGPT the honor.

Before either of those, we need to understand the question. Humanity uses the word "consciousness" for far too many things right now: some of these things, AI have had for a long time; others of which can never be had by anything, including humans.

Re: The Problem with AI Welfare

#154

Post by ben_w »

wat10000 wrote: post #27
gavinray wrote: post #5

Earlier quoted context omitted.

What is so absurd about the idea of ascribing consciousness to some potential future form of AI? We don't understand consciousness, but we've an idea that it's an emergent phenomena. Given the recent papers about how computationally dense our DNA are, and the computing capacity of our brains, is it so unreasonable to assume that a sufficiently complex program running on non-organic matter could give rise to conscious…

It is absurd, but consciousness is fundamentally absurd. Why would doing a bunch of basic arithmetic produce an entity that can experience things the way we do? There's no connection between those two concepts, aside from the fact that the one thing we know that can experience these things is also able to perform computation. But there's no indication that's anything other than a coincidence, or that the causation do…

> Normal biological processes don't seem to do it, it's something particular about the brain. So it's either something that only the brain does, which seems to be something at least vaguely like computation, or the brain is just a conduit and consciousness comes from something functionally like a "soul."

Another option is hidden the first sentence: "Normal biological processes don't *seem* to do it." — emphasis on "seem", because not only is it currently beyond us to have meaningful two-way conversations with dogs about their experiences and as them if they're conscious, we absolutely can't do it with trees or bacteria or our own livers.

We can talk with LLMs, but we also know they're making stuff up, so we can't trust that they're not just saying what got them an up-vote in RLHF training.

Re: The Problem with AI Welfare

#155

Post by vonneumannstan »

bicepjai wrote: post #146
vonneumannstan wrote: post #112

>Or coming from the other way, human consciousness is nothing but an algorithm running on our brains, somehow. You can just stop reading after this. Physicalism is the only realistic framework for viewing consciousness. Everything else is nonsensical.

So what happens when llm has a body that can interact with the world ?

I don't necessarily buy into the idea that you need embodiment in order to get consciousness. Brain in a vat is a thing.

Re: The Problem with AI Welfare

#156

Post by jononor »

vonneumannstan wrote: post #137
jononor wrote: post #122

Earlier quoted context omitted.

There is no LLM suffering today. There is no evidence that there will be such suffering this year. I have seen no credible claims that it is probable to exist this decade, or ever, for that matter. This is not an issue we need to prioritize now.

>There is no LLM suffering today. There's some evidence in favor of LLM suffering. They say they are suffering. Its not proof but its not 'no evidence' either. >There is no evidence that there will be such suffering this year. I have seen no credible claims that it is probable to exist this decade, or ever, for that matter. Your claim actually is the one that is unsupported. Given current trajectories it's likely LLM…

Nothing an LLM says can in itself, right now, be used as evidence of what they 'feel'. It is not established that there is any linking of their output to anything else than the training process (data, loss function, optimizer, etc.). And definitely not to qualia.

On the other hand, it is well know that we can (and commonly do) make them come up with any output we choose. And that their general tendency is to regurgitate any kind of sequence that occurs sufficiently often in the training data.

Re: The Problem with AI Welfare

#157

Post by bbor »

wat10000 wrote: post #150
bbor wrote: post #147

Earlier quoted context omitted.

But what are you experiencing? Something that cannot be defined? If so, do you see the issue there?

I'm not sure. What is "the issue" exactly? The difficulty in defining it certainly makes it hard to talk about. And it makes it impossible to even conceive of how one might detect this phenomenon in other people, or even come up with any sort of theoretical framework around it. But if "the issue" is that this difficulty means I can't really be sure it's even there, no. As I said, this is literally the only thing I ca…

  As I said, this is literally the only thing I can be 100% sure exists.
How can you be 100% confident something exists if you don't even know what it is? That's literally impossible, on a logical level. You can't hold a belief about a concept you don't have -- it would be like a pointer to memory that doesn't exist (i.e. useless, invalid, and erroneous).

Certainly you're aware of things. There are some relevant phenomenological concepts that you hold beliefs about, just like there were real symptoms being described by the Humours system. But you have no justification for bundling them all together into something called "consciousness", which coincidentally comes packed with other, completely unproven assertions.

Re: The Problem with AI Welfare

#158

Post by wat10000 »

bbor wrote: post #157
wat10000 wrote: post #150

Earlier quoted context omitted.

I'm not sure. What is "the issue" exactly? The difficulty in defining it certainly makes it hard to talk about. And it makes it impossible to even conceive of how one might detect this phenomenon in other people, or even come up with any sort of theoretical framework around it. But if "the issue" is that this difficulty means I can't really be sure it's even there, no. As I said, this is literally the only thing I ca…

As I said, this is literally the only thing I can be 100% sure exists. How can you be 100% confident something exists if you don't even know what it is? That's literally impossible, on a logical level. You can't hold a belief about a concept you don't have -- it would be like a pointer to memory that doesn't exist (i.e. useless, invalid, and erroneous). Certainly you're aware of things. There are some relevant phenom…

"Certainly you're aware of things."

Seems you agree with me, then. The "you" i.e. me being aware is what I'm certain exists. I'm not sure what other assertions you think are bundled in there, seems like just the one thing to me.

Re: The Problem with AI Welfare

#159

Post by vonneumannstan »

jononor wrote: post #156
vonneumannstan wrote: post #137

Earlier quoted context omitted.

>There is no LLM suffering today. There's some evidence in favor of LLM suffering. They say they are suffering. Its not proof but its not 'no evidence' either. >There is no evidence that there will be such suffering this year. I have seen no credible claims that it is probable to exist this decade, or ever, for that matter. Your claim actually is the one that is unsupported. Given current trajectories it's likely LLM…

Nothing an LLM says can in itself, right now, be used as evidence of what they 'feel'. It is not established that there is any linking of their output to anything else than the training process (data, loss function, optimizer, etc.). And definitely not to qualia. On the other hand, it is well know that we can (and commonly do) make them come up with any output we choose. And that their general tendency is to regurgit…

If you're harping on 'stochastic parrot' ideas you're just behind the times. Even the most ardent skeptics like Yann Lecun or Gary Marcus don't even believe that nonsense.

Re: The Problem with AI Welfare

#160

Post by jononor »

vonneumannstan wrote: post #159
jononor wrote: post #156

Earlier quoted context omitted.

Nothing an LLM says can in itself, right now, be used as evidence of what they 'feel'. It is not established that there is any linking of their output to anything else than the training process (data, loss function, optimizer, etc.). And definitely not to qualia. On the other hand, it is well know that we can (and commonly do) make them come up with any output we choose. And that their general tendency is to regurgit…

If you're harping on 'stochastic parrot' ideas you're just behind the times. Even the most ardent skeptics like Yann Lecun or Gary Marcus don't even believe that nonsense.

No, just saying that a claim of qualia would require some sort of evidence or methodical argument. And that LLM outputs professing feelings or other state-of-mind like things should by default be assumed to be explained by that the training process (perhaps inadvertently) optimized for such output. Only if such an explanation fails, and another explanation is materially better, should it be considered seriously. Do we have such candidates today?
Post reply on HN