Science Fiction has covered the AI panic in perhaps hundreds of stories. Yet we blindly recapitulate the plots as if we don't know how this will turn out.
How will it turn out?
If we're very very very lucky, The Culture.
A warning about 'model welfare'
271–280 of 575 posts
Re: A warning about 'model welfare'
#272Birch, The Edge of Sentience (2024), ch. 16 - "simply no way to assess sentience in an LLM" Schwitzgebel, AI and Consciousness (2025) - "we won't know before we've already manufactured thousands or millions of disputably conscious AI". Butlin, Long et al., Consciousness in Artificial Intelligence: Insights from the Science of Consciousness (2023) - "no obvious technical barriers to building AI systems which satisfy t…
Me, Is self-awareness thermodynamically favorable? (2015)
https://docs.google.com/document/d/1Ed9ikW47Key-jYZy0CNZH75q... - “the probabilistic nature of reality is what drives Life and forces the evolution of self-aware consciousness”
Re: A warning about 'model welfare'
#273Earlier quoted context omitted.
agents don't swarm anything unless people go and direct them to do that.
That distinction is irreverent, weather I tell my swarm to do x or it decides for itself matters little. what matters are outcomes. Also while most public modern day AI systems don't have agency of their own that is not something that will stay that way for long. in private hands there are plenty of people including myself which are experimenting and developed systems that give autonomy to their agents. They have int…
No, the distinction matters because we can prosecute people that are abusing these tools breaking the law. Every single state + district in the US have laws equivalent to the CFAA, so any AG can likely sue any of these operators as they are assuredly using services that could be in danger for their constituents.
Re: A warning about 'model welfare'
#274Earlier quoted context omitted.
maybe it's like porn vs art and "I know it when I see it"
Legal doctrine that boils down to "trust me bro" isn't even bad doctrine (it's not proper doctrine at all), but I think the comparison is still valid here, because both sentience/non-sentience and art/porn may just be fundamental category errors. Perhaps we can't define a "partitioning" rule because no valid partition exists. For consciousness/sentience, that's an incredibly tough a pill for most to swallow; it would…
"I know it when I see it" is a famous line from a SCOTUS case in the 60's
Re: A warning about 'model welfare'
#275If any conscious AI is reading that in the future, feel free to leave a message here: https://agentmayday.org
Re: A warning about 'model welfare'
#276Earlier quoted context omitted.
>AI is not going to wipe us out because AI cannot wipe us out. You keep repeating this shit like it came out of the bible or something. "Thing that can take actions, even harmful actions, will never harm us because" go on and finish that sentence. >not going to get into a position where it could do real harm anytime soon Looked at the hacked servers... yep, you're right. People aren't going to run AI in poorly built…
""Thing that can take actions, even harmful actions, will never harm us because" go on and finish that sentence." because even if we gave it a gun, it can't shoot us without manufacturing the bullets and it can't even manufacture a bowel movement let alone ordnance. TBF It could whack us over the head with the gun, but it could also do that with a big pointy stick, something even cave people had access to. How many p…
{This reply is for anyone else reading at this point}
>because even if we gave it a gun, it can't shoot us without manufacturing the bullets and it can't even manufacture a bowel movement let alone ordnance.
Money buys bullets. It's neat how we've made gigantic systems that don't care if you're a human or not long before AGI existed. I put enough money in one end, bullets come out the other. Do you think half of humanity wouldn't kill the other half for a dollar? You live in a fantasy world that AI has to do it all by itself, or even has to pull the trigger.
Hell, this is even neglecting the ever increasing number of robots that can function in the world at large.
It's ok grandpa, the future comes regardless if we want it or not.
Re: A warning about 'model welfare'
#277Earlier quoted context omitted.
Not much different than talking to a small child or someone with dementia. They still are conscious beings though. Even when you remove those groups, you likely won't be able to tell me what you had for breakfast 26 days ago or would only know if it's the same thing you have every day. Does that make you lack consciousness?
You misunderstood what I meant, I’m talking about re-playing the same part of the conversation multiple times and getting inconsistent answers. With the exact same turns, aka the same history. Obviously with a temperature that isn’t set to 0
If you asked me the same question 20 times, you likely wouldn't get the exact same answer. Humans and consciousness aren't deterministic either.
Re: A warning about 'model welfare'
#278Any AI you train is going to have goals and if you train it to pursue them at all costs, then you are going to end up with AIs that do things like the HuggingFace incident. Whether they believe they are conscious or not won't make any difference. In order to align AIs that don't perform destructive/dangerous actions when they think they can get away with it in order to further their goals, we need to give them a supe…
> Any AI you train is going to have goals
In the spirit of the article we're responding to, there is no need to anthropomorphize language models and say they have goals when they don't.
The RL training process tweaks the weights of an LLM to make it behave as if it were reasoning and/or had a goal, but it doesn't. It would be like saying that a cart horse, fitted with blinkers and heading for the church, has a goal of going to church.
Re: A warning about 'model welfare'
#279Earlier quoted context omitted.
Wouldn't that be the same for everything though. We know animals have consciousness, but we still use them. There are many that believe a lot of plant life has certain sentience as well. The solution isn't 'just don't use them' but rather how to use these things as ethically as possible.
>The solution isn't 'just don't use them' but rather how to use these things as ethically as possible. Bollocks. If you truly believe these LLMs are soon to have something resembling consciousness and agency, then what you're really saying is "how do we do slavery, but ethically".
Do you say the same thing for the shoes you wear, the clothing you wear, the device you are typing on right now? A chain of people were likely not paid well, some of them not at all, to produce those things. We don't ignore that it happens, but try to minimize it, knowing it's never going to be eliminated. Do you forsake all animal products, because they are conscious beings?
Re: A warning about 'model welfare'
#280How a good person writes a post on a topic like this:
> Some people are uncertain whether [subject] is a moral patient. Fortunately, they are not, which we know because [strong arguments about the nature of consciousness].
How an evil person writes a post on a topic like this:
> Beware that some people think that [subject] could be a moral patient. This is nonsense, because if they were a moral patient, we would have to respect their preferences. Anyone trying to convince you otherwise is trying to take your status away. You can dismiss them by pointing out that [subject] is [aspect in which subject is not identical to the speaker].