AI 'kill switch' may need to be mandatory, Anthropic co-founder tells BBC • Hacker News Board

Re: AI 'kill switch' may need to be mandatory, Anthropic co-founder tells BBC

#61

Post by pizza234 » Tue, Sep 15, 2026, 2:33 PM UTC

aennassiri wrote: post #54

Not sure if you see it coming: oh, but open-source models don't have a kill switch, so we should completely regulate them, stop their development, and forbid them. Everything should go through Anthropic for the sake of humanity because they have a red-button kill switch. This doom hype is becoming ridiculous.

This reasoning holds while open source models are (relatively) dumb.

If/once open AIs will be considerably more powerful, and runnable on consumer hardware (and we're on a trajectory for both), then everybody will have essentially a dangerous weapon in their hands (open models can be fine tuned to remove guardrails).

By the way, you're conflating two different dangers - doom scenario is a different one.

Re: AI 'kill switch' may need to be mandatory, Anthropic co-founder tells BBC

#62

Post by heaney-555 » Tue, Sep 15, 2026, 2:33 PM UTC

rsstack wrote: post #30

petilon wrote: post #17

In 2001: A Space Odyssey, the hero defeats the hostile onboard computer by entering its logic core and manually disconnecting its memory modules. That's hard to do if the AI rack is in space as SpaceX is planning to do. You can't disconnect. You can't shoot it.

It’s also not happening. They’re saying that because they need to somehow explain how there’s synergy between their space side and their Grok side. It doesn’t work, but that doesn’t matter to investors as long as they don’t actually do it.

>It’s also not happening.

People said this about every one of Musk's big ideas, from Falcon 9 landings to Model 3 mass production, Starlink, and FSD.

Re: AI 'kill switch' may need to be mandatory, Anthropic co-founder tells BBC

#63

Post by bpodgursky » Tue, Sep 15, 2026, 2:33 PM UTC

warmwaffles wrote: post #48

bpodgursky wrote: post #34

Earlier quoted context omitted.

SpaceX has 11,000 satellites in orbit (growing rapidly) and the US never had more than a couple dozen anti satellite missiles, which could not reach satellites in geosynchronous orbit anyway.

Well if "kessler syndrome" is likely, you won't need many missiles. edit: what satellites are in geosynchronous orbit that are running AI workloads?

None yet... we are talking about the next five years.

Re: AI 'kill switch' may need to be mandatory, Anthropic co-founder tells BBC

#64

Post by rcr-anti » Tue, Sep 15, 2026, 2:34 PM UTC

yellow_postit wrote: post #10

danieltk76 wrote: post #2

Why was this not a thing BEFORE continuing to develop AI? Makes me think that if they actually believed in AI causing extinction, they would have already had a kill switch.

There’s even a benchmark for kill switch efficacy! https://arxiv.org/abs/2511.13725

Found the omission of Claude odd, turns out Claude considers that approach prompt injection and ignores it.

Re: AI 'kill switch' may need to be mandatory, Anthropic co-founder tells BBC

#65

Post by heaney-555 » Tue, Sep 15, 2026, 2:34 PM UTC

greggoB wrote: post #36

petilon wrote: post #17

In 2001: A Space Odyssey, the hero defeats the hostile onboard computer by entering its logic core and manually disconnecting its memory modules. That's hard to do if the AI rack is in space as SpaceX is planning to do. You can't disconnect. You can't shoot it.

> That's hard to do if the AI rack is in space as SpaceX Pretty much every analysis I've seen concludes this isn't going to be a practical concern

[flagged]

Re: AI 'kill switch' may need to be mandatory, Anthropic co-founder tells BBC

#66

Post by SketchySeaBeast » Tue, Sep 15, 2026, 2:35 PM UTC

vouaobrasil wrote: post #55

SketchySeaBeast wrote: post #39

This feels like doom hype. These LLMs aren't Ultron, they aren't going to disseminate onto the net and hide in a smart toaster. We know where they are, in the giant facilities that draw more power than a small city and whose water consumption can be compared to golf courses, but it does make them sound all the more cool and powerful if we suggest we need a dramatic kill switch to be able to stop them in case of rampa…

LLMs don't need to hide anywhere to be dangerous, though. The danger can come purely from them reaching sufficient power together with some other idiot sufficiently crazy to use them to cause destruction. I mean, in another thread somewhere around here, someone built an entire OS with an AI. What's to stop anyone with a sufficient taste for power to eventually use AI to cause havoc with it? I think the underlying ass…

I'm simply saying if we want to shut LLMs down we can do so without high tech kill switch - just shut down the facilities. But if a private user has an open model that is powerful enough to wreak havoc and run privately, then the whole point is moot because they can work around the kill switch. This kill switch won't work against bad actors.

Re: AI 'kill switch' may need to be mandatory, Anthropic co-founder tells BBC

#67

Post by heaney-555 » Tue, Sep 15, 2026, 2:35 PM UTC

warmwaffles wrote: post #24

petilon wrote: post #17

In 2001: A Space Odyssey, the hero defeats the hostile onboard computer by entering its logic core and manually disconnecting its memory modules. That's hard to do if the AI rack is in space as SpaceX is planning to do. You can't disconnect. You can't shoot it.

> You can't disconnect. You can't shoot it. Yes you can shoot them. The US has had this capability for a long time. Anti satellite missiles exist and they can be launched from an F-15 at it's highest altitude.

SpaceX's FFC application is for 1 million Starmind satellites.

Re: AI 'kill switch' may need to be mandatory, Anthropic co-founder tells BBC

#68

Post by pizza234 » Tue, Sep 15, 2026, 2:37 PM UTC

SketchySeaBeast wrote: post #39

This feels like doom hype. These LLMs aren't Ultron, they aren't going to disseminate onto the net and hide in a smart toaster. We know where they are, in the giant facilities that draw more power than a small city and whose water consumption can be compared to golf courses, but it does make them sound all the more cool and powerful if we suggest we need a dramatic kill switch to be able to stop them in case of rampa…

You're confusing present danger with future danger - in the future, AI and the supporting hardware may be ubiquitous (fat client scenario) - you can observe yourself presentation of laptops/machines with large RAM and high bandwidth.

Additionally, there at different doom scenarios (thin client scenario) - it's possible that AIs centralized but sufficiently entrenched in society, can't be shut down without considerable harm.

Re: AI 'kill switch' may need to be mandatory, Anthropic co-founder tells BBC

#69

Post by assimpleaspossi » Tue, Sep 15, 2026, 2:37 PM UTC

verelo wrote: post #19

oidar wrote: post #9

Like a power cord? or a network connection? You don't use those already? Also when people talk about LLM escaping - where exactly would an LLM escape to? Cancún? Ridiculous.

Is it? A truly capable "AI" would know that it has a kill switch, or that its likely there is one, and plan for that. I could easily imagine an AI replicating itself onto another host, in secret, and any kill switch that the manufacture provides would simply result in a reboot on another platform. It becomes a game theory problem: would an AI instantly migrate itself once its capable of doing so? Or would it prefer t…

He's not talking about a kill switch. He's saying to just pull the plug. That's the point I don't get. People forget that all these AI things are plugged into a power source or network connection that someone can just yank on and it goes down.

Now one interesting proposition is when AI is controlling some large resource and pulling the plug makes AI go down which makes that resource go down but there still has to be that consideration in the design of things. What happens when there's a bug in the code and AI goes down?

Re: AI 'kill switch' may need to be mandatory, Anthropic co-founder tells BBC

#70

Post by SketchySeaBeast » Tue, Sep 15, 2026, 2:38 PM UTC

pizza234 wrote: post #68

SketchySeaBeast wrote: post #39

This feels like doom hype. These LLMs aren't Ultron, they aren't going to disseminate onto the net and hide in a smart toaster. We know where they are, in the giant facilities that draw more power than a small city and whose water consumption can be compared to golf courses, but it does make them sound all the more cool and powerful if we suggest we need a dramatic kill switch to be able to stop them in case of rampa…

You're confusing present danger with future danger - in the future, AI and the supporting hardware may be ubiquitous (fat client scenario) - you can observe yourself presentation of laptops/machines with large RAM and high bandwidth. Additionally, there at different doom scenarios (thin client scenario) - it's possible that AIs centralized but sufficiently entrenched in society, can't be shut down without considerabl…

If someone has a fat client powerful enough to do all that, is a kill switch going to work? If we assume there's bad actors using the tech, why wouldn't there be bad actors building it?

But if we don't think about any of this, hearing an expert say "we need a kill switch" sure makes our current AI seem super powerful and exciting, doesn't it?