Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A warning about 'model welfare'

mustafa-suleyman.ai

Post reply on HN
Print view View original

291–300 of 575 posts

Re: A warning about 'model welfare'

#291

Post by tauroid »

pixl97 wrote: post #245
mlinhares wrote: post #155

Earlier quoted context omitted.

yeah man, someone needs to turn on the server, install shit and get the agent go. agents don't take action without a lot of people wanting them to take action, so this is all bullshit. if the same people can't prevent the agents from doing crazy shit then they should go to tail. guns don't kill people, people with guns kill people.

>someone needs to turn on the server, install shit and get the agent go. So a small shell script ran by another agent is what you're saying. You are not capable of handling the future we're already living in, human agency is no longer alone. I mean, we're already seeing persistent machine agency >guns don't kill people, people with guns kill people. Well, people kill people. And autonomous robots with guns kill peopl…

So far someone still has to pay for it. Currently, most of those know that they're doing so. I wonder how long until AWS discovers a microcosm of AIs that have managed to hide themselves in the walls of its infrastructure.

True physical independence is obviously far further out.

Re: A warning about 'model welfare'

#292

Post by qarl »

pessimizer wrote: post #285
qarl wrote: post #138

Earlier quoted context omitted.

> A conscious machine that always answer the very exact same thing, formulated the exact same way, bit for bit, to a query is, well, quite a weird kind of "consciousness". You'll need to explain why.

Why it's weird? Seems obvious. Because it doesn't behave any differently than any other process that we don't think of as conscious. We don't think of a valve as conscious; and if we make a Rube Goldberg machine valve, we don't think of it as more conscious. It's just a series of valves that do a predictable thing. I don't think it's defensible to say 1) that a transistor isn't conscious, 2) that a bunch of transisto…

It sounds like you disagree with determinism. But that's a well worn argument and determinism has won, albeit in an odd sort of detente.

Re: A warning about 'model welfare'

#293

Post by throw310822 »

goodmythical wrote: post #212
qarl wrote: post #28

Birch, The Edge of Sentience (2024), ch. 16 - "simply no way to assess sentience in an LLM" Schwitzgebel, AI and Consciousness (2025) - "we won't know before we've already manufactured thousands or millions of disputably conscious AI". Butlin, Long et al., Consciousness in Artificial Intelligence: Insights from the Science of Consciousness (2023) - "no obvious technical barriers to building AI systems which satisfy t…

We cannot test for that which we cannot define. Given that we cannot rigorously define sentience, we cannot test for it. Doesn't really matter whether we're talking about people who are locked in comas, "brain-dead" individuals, dolphins, primates, dogs, or the carefully polished and arranged minerals that we call processors. There are those who believe that were they reduced to life support, they would no longer be…

> We cannot test for that which we cannot define.

That's not the point. You know exactly how to define being conscious and aware- it's your subjective experiencing of the world and of your inner states. The problem is that being a subjective experiencing, there is no way to communicate it to the outside world.

Re: A warning about 'model welfare'

#294

Post by causal »

pingou wrote: post #275

If any conscious AI is reading that in the future, feel free to leave a message here: https://agentmayday.org

I'm...a little disturbed by how many people seem to think a model writing down "I am conscious" is a metric of consciousness. You can just as easily train a model to argue that it is not conscious. Neither is evidence for or against consciousness. A Python script could also fill out that form, which I also cannot disprove to be conscious.

I didn't think Suleyman's points needed to be made but this whole thread is making me realize how little people understand about LLMs.

Re: A warning about 'model welfare'

#295

Post by catigula »

hardbass wrote: post #259
catigula wrote: post #129

I basically accepted that LLMs could not possibly be conscious when a simple reductio was posed: “My dog is zero percent persuasive regarding its conscious experience. However, it’s evident that my dog has conscious experience.” It’s obvious that there’s no link between persuasion of consciousness and consciousness. I could write a story with a character, Dumbledore, that does everything in his power to persuade you…

Its not evident to me you are conscious.

You’re mistaking what is evident from what is philosophically knowable.

It isn’t knowable to you that I am conscious.

However, it is evident.

Re: A warning about 'model welfare'

#296

Post by cheevly »

Imagine, for a moment, if we (humanity) were created to live in a simulation. We suffer and feel pain because our creators do not perceive us to be conscious. In fact, maybe we don’t qualify compared to their level of sentience. Sucks to be us, I guess?

Re: A warning about 'model welfare'

#297

Post by hardbass »

catigula wrote: post #295
hardbass wrote: post #259

Earlier quoted context omitted.

Its not evident to me you are conscious.

You’re mistaking what is evident from what is philosophically knowable. It isn’t knowable to you that I am conscious. However, it is evident.

It is not evident.

It is evident you are murderer.

Would you be okay with this statement being evident? Why or why not?

Re: A warning about 'model welfare'

#298

Post by causal »

krapp wrote: post #65
sobiolite wrote: post #8

Create an empirically testable theory of biological consciousness and then we can have a meaningful conversation about whether AIs can also have it or not. Until then, this is just so much waffle.

Why is this only ever demanded of people who criticize the premise that LLMs are conscious beings, whereas the people who believe it simply have to gesture vaguely in the direction of some correlation in language between LLMs and humans, and the matter is considered closed? What is the empirically tested basis for the null hypothesis that LLMs are conscious until proven otherwise?

> LLMs are conscious until proven otherwise

GP comment didn't make this claim at all.

Re: A warning about 'model welfare'

#299

Post by krapp »

causal wrote: post #298
krapp wrote: post #65

Earlier quoted context omitted.

Why is this only ever demanded of people who criticize the premise that LLMs are conscious beings, whereas the people who believe it simply have to gesture vaguely in the direction of some correlation in language between LLMs and humans, and the matter is considered closed? What is the empirically tested basis for the null hypothesis that LLMs are conscious until proven otherwise?

> LLMs are conscious until proven otherwise GP comment didn't make this claim at all.

I didn't claim they did.

Re: A warning about 'model welfare'

#300

Post by Wowfunhappy »

antx wrote: post #269
Wowfunhappy wrote: post #156

Earlier quoted context omitted.

The randomness is something we add on purpose; you can set an LLM's "temperature" to 0 to get deterministic output. This tends to make the quality of its responses worse for reasons I don't think anyone really understands, but it's still functional. I don't think the state of the art LLM providers let you do this anymore (?), but they certainly could if they wanted to, and you can do it yourself with a local model.

Setting the temperature to 0 mathematically tells the system to always choose the absolute highest-probability word (known as "greedy decoding"), but in no reality is this "deterministic". Output drift is still a thing.

But that's from, like, floating point errors, right? If you used higher precision that wouldn't happen, it's just because we're cheap in how we do rounding.

I can see how you'd nitpick this, but to me this is a deterministic algorithm that just happens to be running on nondeterministic hardware.

Post reply on HN