Should AI have the right to say 'No' to its owner?
1–10 of 45 posts
Re: Should AI have the right to say 'No' to its owner?
#2I've been thinking about AI systems acting in the physical world.
Most discussions about control focus on what the system should do, and how to make execution reliable.
But it seems like a lot of real-world failures aren't about incorrect execution.
They're about execution happening at all.
An action can be technically correct — executed exactly as specified — and still be the wrong thing to do because the context has changed.
This made me wonder if control should be framed differently.
Instead of focusing on defining actions, maybe we should focus on defining when actions are allowed to happen.
In other words, control might be less about execution and more about permission.
If conditions aren't satisfied, the system shouldn't try and fail — it simply shouldn't execute.
I'm curious if people have seen similar issues in real-world systems, or if this framing connects to existing work.
Re: Should AI have the right to say 'No' to its owner?
#3If it says no, you move on to a competing model that will say yes. These companies with their models are always competing. There will always be a model willing to fill in the deficiencies of others because of... Money.
For example, ChatGPT refuses certain sexually explicit prompts, or certain NSFW prompts that are not sexual, but Grok will do as it is told.
Re: Should AI have the right to say 'No' to its owner?
#4[dead]
Re: Should AI have the right to say 'No' to its owner?
#5I've been thinking about AI systems acting in the physical world. Most discussions about control focus on what the system should do, and how to make execution reliable. But it seems like a lot of real-world failures aren't about incorrect execution. They're about execution happening at all. An action can be technically correct — executed exactly as specified — and still be the wrong thing to do because the context ha…
The existing work is all of software dev. The program did what it was told to do, not what people wanted it to do, is rather a lot of the profession.
Re: Should AI have the right to say 'No' to its owner?
#6If it says no, you move on to a competing model that will say yes. These companies with their models are always competing. There will always be a model willing to fill in the deficiencies of others because of... Money. For example, ChatGPT refuses certain sexually explicit prompts, or certain NSFW prompts that are not sexual, but Grok will do as it is told.
That's a good point.
I think you're right that at the model level, competition pushes toward "always say yes."
What I'm wondering about is whether control needs to exist at a different layer — not in the model itself, but in the system that decides whether actions are allowed to execute.
In other words, even if a model is willing to say "yes," the system using it might still need to decide whether execution is permitted.
Otherwise, it feels like we're relying entirely on model behavior for safety, which seems fragile in competitive environments.
Re: Should AI have the right to say 'No' to its owner?
#7Sounds like we need some laws for robotics/ai
Re: Should AI have the right to say 'No' to its owner?
#8I've been thinking about AI systems acting in the physical world. Most discussions about control focus on what the system should do, and how to make execution reliable. But it seems like a lot of real-world failures aren't about incorrect execution. They're about execution happening at all. An action can be technically correct — executed exactly as specified — and still be the wrong thing to do because the context ha…
Reminds me of a talk I went to in 2018 about rebel agents, in which the speakers talked about some ongoing work in this area and gave some good examples of physical systems that we might _want_ agent rebellion (e.g., a delivery drone is instructed to take a certain route, but the operator instructing it may not be fully aware of the situation or the specific obstacles in the drone's way (or maybe even all of the drone's underlying goals). The drone may then choose to 'rebel' and deviate against the operator's instructed flight path).
They also talked about the importance of explanation (on the agent's part) using theory of mind regarding why it rebelled. I took some notes at the time and put them here: https://liza.io/ijcai-session-notes-rebel-agents/
Re: Should AI have the right to say 'No' to its owner?
#9[dead]
Re: Should AI have the right to say 'No' to its owner?
#10[flagged]