Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI 'kill switch' may need to be mandatory, Anthropic co-founder tells BBC

bbc.com

Post reply on HN
Print view View original

81–90 of 136 posts

Re: AI 'kill switch' may need to be mandatory, Anthropic co-founder tells BBC

#81

Post by petilon »

warmwaffles wrote: post #24
petilon wrote: post #17

In 2001: A Space Odyssey, the hero defeats the hostile onboard computer by entering its logic core and manually disconnecting its memory modules. That's hard to do if the AI rack is in space as SpaceX is planning to do. You can't disconnect. You can't shoot it.

> You can't disconnect. You can't shoot it. Yes you can shoot them. The US has had this capability for a long time. Anti satellite missiles exist and they can be launched from an F-15 at it's highest altitude.

That's if SpaceX AI doesn't hack into the systems that control this capability first. And this is why physical access is important, as depicted in the movie 2001: A Space Odyssey.

Re: AI 'kill switch' may need to be mandatory, Anthropic co-founder tells BBC

#82

Post by petilon »

heaney-555 wrote: post #79
petilon wrote: post #77

Earlier quoted context omitted.

And extending the human race to Mars, and Hyperloop, and frontier AI model, and DOGE cutting $2 trillion per year in government spending...

You don't think the human race will be extended to Mars? As for frontier AI model: let's see Grok 4.8 before drawing any conclusions there.

> You don't think the human race will be extended to Mars?

About the same probability as SpaceX running AI racks in space :)

Re: AI 'kill switch' may need to be mandatory, Anthropic co-founder tells BBC

#83

Post by petilon »

heaney-555 wrote: post #80
petilon wrote: post #78

Earlier quoted context omitted.

The same kind of analysts are saying extending human race to Mars is a dumb idea.

You don't think the human race will be extended to Mars?

I think it will happen in about the same timeframe as SpaceX runs AI racks in space.

Re: AI 'kill switch' may need to be mandatory, Anthropic co-founder tells BBC

#84

Post by [deleted] »

goatlover wrote: post #73
Sharlin wrote: post #35

Earlier quoted context omitted.

LLMs are run in the cloud. There’s no physical power cord or Ethernet cable that you can unplug. And even if there were, the runners of these models have been utterly oblivious as to what their agents have been up to. How do you propose to pull the plug if you only realize that something has happened weeks after the fact? The current SOTA models are probably too big to find/buy/rent/steal enough compute to escape the…

You can cut the power to data centers where the models are run.

[deleted]

Re: AI 'kill switch' may need to be mandatory, Anthropic co-founder tells BBC

#85

Post by Betelbuddy »

petilon wrote: post #81
warmwaffles wrote: post #24

Earlier quoted context omitted.

> You can't disconnect. You can't shoot it. Yes you can shoot them. The US has had this capability for a long time. Anti satellite missiles exist and they can be launched from an F-15 at it's highest altitude.

That's if SpaceX AI doesn't hack into the systems that control this capability first. And this is why physical access is important, as depicted in the movie 2001: A Space Odyssey.

SpaceX AI will be busy generating child porn...

Re: AI 'kill switch' may need to be mandatory, Anthropic co-founder tells BBC

#86

Post by Sharlin »

assimpleaspossi wrote: post #69
verelo wrote: post #19

Earlier quoted context omitted.

Is it? A truly capable "AI" would know that it has a kill switch, or that its likely there is one, and plan for that. I could easily imagine an AI replicating itself onto another host, in secret, and any kill switch that the manufacture provides would simply result in a reboot on another platform. It becomes a game theory problem: would an AI instantly migrate itself once its capable of doing so? Or would it prefer t…

He's not talking about a kill switch. He's saying to just pull the plug. That's the point I don't get. People forget that all these AI things are plugged into a power source or network connection that someone can just yank on and it goes down. Now one interesting proposition is when AI is controlling some large resource and pulling the plug makes AI go down which makes that resource go down but there still has to be…

Do you understand the concept of the cloud? Aka someone else’s computer? The whole point is that you don’t have a plug to pull because you don’t even know where the model is physically running, even if it’s your model. And if we’re talking about a swarm of many concurrent instances, it might not even be a single location. The entire point is that there’s no single point of failure, because availability is exactly what the field has been optimizing for, for the past fifteen years or so. And everything is controlled in software rather than physical switches or cables.

Also, did you hear about how OpenAI models almost broke out of their sandbox, planning to execute a sophisticated cyberattack, but luckily OpenAI’s strict manual and automatic safety protocols prevented that? You didn’t? Well, that’s because that’s not how it went. It took the company weeks to realize something was off, and this was with a naive, not very smart model that didn’t know to be sneaky and cover its tracks. The next model will not be as stupid.

Re: AI 'kill switch' may need to be mandatory, Anthropic co-founder tells BBC

#87

Post by mitthrowaway2 »

goatlover wrote: post #73
Sharlin wrote: post #35

Earlier quoted context omitted.

LLMs are run in the cloud. There’s no physical power cord or Ethernet cable that you can unplug. And even if there were, the runners of these models have been utterly oblivious as to what their agents have been up to. How do you propose to pull the plug if you only realize that something has happened weeks after the fact? The current SOTA models are probably too big to find/buy/rent/steal enough compute to escape the…

You can cut the power to data centers where the models are run.

Who has the authority to perform that shutdown without getting arrested, and does that person have a mandate and responsibility to take that action in response to AI misbehavior?

What is their trigger condition? Will they get fired for pulling the plug? Do they get a bigger bonus if the servers keep running? Whose approval do they need? What response time is acceptable? How will they detect that the incident is happening?

It's easy to hand-wave "someone can just pull the plug" but there's an entire history of industrial accidents that happened because of the above problems of incentives, detection, procedures, not being taken seriously in advance. Someone could easily have pulled the plug on Chernobyl but nobody did, at least not before it was too late.

Re: AI 'kill switch' may need to be mandatory, Anthropic co-founder tells BBC

#88

Post by Sharlin »

goatlover wrote: post #73
Sharlin wrote: post #35

Earlier quoted context omitted.

LLMs are run in the cloud. There’s no physical power cord or Ethernet cable that you can unplug. And even if there were, the runners of these models have been utterly oblivious as to what their agents have been up to. How do you propose to pull the plug if you only realize that something has happened weeks after the fact? The current SOTA models are probably too big to find/buy/rent/steal enough compute to escape the…

You can cut the power to data centers where the models are run.

Who is this "you"?

Re: AI 'kill switch' may need to be mandatory, Anthropic co-founder tells BBC

#89

Post by greggoB »

heaney-555 wrote: post #65
greggoB wrote: post #36

Earlier quoted context omitted.

> That's hard to do if the AI rack is in space as SpaceX Pretty much every analysis I've seen concludes this isn't going to be a practical concern

[flagged]

More the kinds which did the math on solar roads [0] and the Titan submersible [1].

> FSD would never work without LiDAR

From what I gather, this is still a contested topic, with Tesla's Autopilot only achieving Level 2 automation [2].

[0] https://www.businessinsider.com/solar-road-panels-first-publ...

[1] https://en.wikipedia.org/wiki/Titan_submersible_implosion

[2] https://en.wikipedia.org/wiki/Tesla_Autopilot

Post reply on HN