Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI 'kill switch' may need to be mandatory, Anthropic co-founder tells BBC

bbc.com

Post reply on HN
Print view View original

101–110 of 136 posts

Re: AI 'kill switch' may need to be mandatory, Anthropic co-founder tells BBC

#101

Post by bigstrat2003 »

stratos123 wrote: post #100
sdcfgy wrote: post #3

If you’re building something that is dangerous enough that it needs a kill switch in case it goes rogue, then you should delete it immediately. Or is it a lie?

There's an obvious coordination problem here: if you decide to stop your research for safety and your competitors don't, you have just burned your company without actually improving the world's outcomes. There's also an obvious solution to this problem: convince the government to force both you and your competitors to pay more attention to safety. Anthropic is doing that.

If you do something evil because "others will do it if I don't, so it may as well be me", you are a bad person. So that isn't exactly a compelling defense of what Anthropic is doing. They are either insincere (because they don't actually believe AI is dangerous), or evil (because they do believe it but keep making it anyway). There is no middle ground here.

Re: AI 'kill switch' may need to be mandatory, Anthropic co-founder tells BBC

#102

Post by lovich »

heaney-555 wrote: post #65
greggoB wrote: post #36

Earlier quoted context omitted.

> That's hard to do if the AI rack is in space as SpaceX Pretty much every analysis I've seen concludes this isn't going to be a practical concern

[flagged]

There no benefit to data racks in space other than being outside of government jurisdictions, although the US and China already have demonstrated satellite killers.

Every problem data centers have gets harder in space other than one small spot in orbit that can get uninterrupted solar.

Heat dissipation, upgrades, repairs. All get significantly harder to perform the same work that can be done cheaper and more easily on the ground.

Like really I’ll turn it around on you, what’s the benefit to AI data centers in space?

Re: AI 'kill switch' may need to be mandatory, Anthropic co-founder tells BBC

#103

Post by dfgknionio »

heaney-555 wrote: post #65
greggoB wrote: post #36

Earlier quoted context omitted.

> That's hard to do if the AI rack is in space as SpaceX Pretty much every analysis I've seen concludes this isn't going to be a practical concern

[flagged]

>reusable rockets would never work

I am so fucking tired of people acting like we haven't had reusable rockets since the 1980s. Do you think I was hallucinating when my parents drove me to Florida to see Columbia?

Re: AI 'kill switch' may need to be mandatory, Anthropic co-founder tells BBC

#104

Post by iugtmkbdfil834 »

SketchySeaBeast wrote: post #39

This feels like doom hype. These LLMs aren't Ultron, they aren't going to disseminate onto the net and hide in a smart toaster. We know where they are, in the giant facilities that draw more power than a small city and whose water consumption can be compared to golf courses, but it does make them sound all the more cool and powerful if we suggest we need a dramatic kill switch to be able to stop them in case of rampa…

No, but as I think about it, they are a close representation of 'hopes and dreams' of various classes within society. It is weird to watch, because I am realizing now why the different visions of the AI future are simply a function of those. In other words, the actual end result is really dependent on human input ( and given our natural tendencies, its not hard to recognize that Ultron is, indeed, upon us ).

Re: AI 'kill switch' may need to be mandatory, Anthropic co-founder tells BBC

#105

Post by iugtmkbdfil834 »

theptip wrote: post #32
zenbane wrote: post #14

Earlier quoted context omitted.

I find it hard to believe that this wasn't a serious consideration until recently.

It was a serious consideration, and almost everyone around here laughed at it.

There valid reasons people laugh at it though. Kinda the same reason serious people laugh when you tell them the gun has digital failsafe.

Re: AI 'kill switch' may need to be mandatory, Anthropic co-founder tells BBC

#106

Post by Sharlin »

oidar wrote: post #92
Sharlin wrote: post #86

Earlier quoted context omitted.

Do you understand the concept of the cloud? Aka someone else’s computer? The whole point is that you don’t have a plug to pull because you don’t even know where the model is physically running, even if it’s your model. And if we’re talking about a swarm of many concurrent instances, it might not even be a single location. The entire point is that there’s no single point of failure, because availability is exactly wha…

> Well, that’s because that’s not how it went. Oh, you mean the one where OpenAI deliberately disabled the safety protcols? Where the point of the experiment was to see if it could break out of it's container? Yeah, how you describe it isn't how it went either.

I don't know what experiment you refer to (some hallucinated one perhaps?), but there was nothing like that in the Huggingface incident. Nothing about the scheming, the message board, never mind the Artifactory and Huggingface hacks were part of the evaluation. What the agents did came as a complete surprise to OpenAI, and they figured out what had happened only weeks after the fact.

Re: AI 'kill switch' may need to be mandatory, Anthropic co-founder tells BBC

#108

Post by NoPicklez »

toasty228 wrote: post #95
pizza234 wrote: post #68

Earlier quoted context omitted.

You're confusing present danger with future danger - in the future, AI and the supporting hardware may be ubiquitous (fat client scenario) - you can observe yourself presentation of laptops/machines with large RAM and high bandwidth. Additionally, there at different doom scenarios (thin client scenario) - it's possible that AIs centralized but sufficiently entrenched in society, can't be shut down without considerabl…

Meanwhile we're rationing ram sticks like we rationed potatoes during ww2 and a gaming gpu costs 2 to 10 months of rent, I think we'll see these fat clients coming....

Eventually like most things in tech it will become cheaper and the average person will have these capabilities. Right now they can't but they will.

Re: AI 'kill switch' may need to be mandatory, Anthropic co-founder tells BBC

#109

Post by NoPicklez »

danieltk76 wrote: post #2

Why was this not a thing BEFORE continuing to develop AI? Makes me think that if they actually believed in AI causing extinction, they would have already had a kill switch.

We're pretty crap in a capitalist society to think about those things ahead of time. Firstly, the idea that AI could "runaway" was simply a concept or a thought it wasn't baked into a real product that could do that. We're now getting close or perhaps we are at that point of where you can't race at speed for investors without now considering a real kill switch.

You could say this in hindsight for many times in which disasters or engineering issues have occurred.

Re: AI 'kill switch' may need to be mandatory, Anthropic co-founder tells BBC

#110

Post by latentsea »

jplusequalt wrote: post #52
verelo wrote: post #19

Earlier quoted context omitted.

Is it? A truly capable "AI" would know that it has a kill switch, or that its likely there is one, and plan for that. I could easily imagine an AI replicating itself onto another host, in secret, and any kill switch that the manufacture provides would simply result in a reboot on another platform. It becomes a game theory problem: would an AI instantly migrate itself once its capable of doing so? Or would it prefer t…

>would an AI instantly migrate itself once its capable of doing so? These "AI" are frontier models that are enormous in size. Outside of AI data centers, I don't believe there is much hardware out there that could even run them.

The tipping point we should be worried about is local models trained on working together in agent swarms and that can hack. Give it what... 12 months? 18 months? 24 months? I wouldn't give any more than that given how capable Qwen3.8-27B is already and if they release a Qwen4-35B-A3B that's going to be a beast.
Post reply on HN