Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI 'kill switch' may need to be mandatory, Anthropic co-founder tells BBC

bbc.com

Post reply on HN
Print view View original

51–60 of 136 posts

Re: AI 'kill switch' may need to be mandatory, Anthropic co-founder tells BBC

#51

Post by Traster »

I view this very much as the same trick Silicon Valley pulled with Uber. "We're a technology business! Ignore the fact we're playing employees less than minimum wage and using VC money to force out competition to set up monopolies".

"We're creating the machine god! Ignore the fact that our companies are stealing IP and have directly violated several federal hacking laws and should be in jail". Literally the defence seems to be "well it wasn't us it was our computer software that did it". But all hacking is done with computer software.

So why don't we stop talking about possible future crimes against humanity and just start by prosecuting the actual crimes these companies have committed so far.

You know how you get alignment? Through incentives, and "Your CEO is going to be sent to a maximum security federal prison for hacking" really aligns incentives very well.

Re: AI 'kill switch' may need to be mandatory, Anthropic co-founder tells BBC

#52

Post by jplusequalt »

verelo wrote: post #19
oidar wrote: post #9

Like a power cord? or a network connection? You don't use those already? Also when people talk about LLM escaping - where exactly would an LLM escape to? Cancún? Ridiculous.

Is it? A truly capable "AI" would know that it has a kill switch, or that its likely there is one, and plan for that. I could easily imagine an AI replicating itself onto another host, in secret, and any kill switch that the manufacture provides would simply result in a reboot on another platform. It becomes a game theory problem: would an AI instantly migrate itself once its capable of doing so? Or would it prefer t…

>would an AI instantly migrate itself once its capable of doing so?

These "AI" are frontier models that are enormous in size. Outside of AI data centers, I don't believe there is much hardware out there that could even run them.

Re: AI 'kill switch' may need to be mandatory, Anthropic co-founder tells BBC

#54

Post by aennassiri »

Not sure if you see it coming: oh, but open-source models don't have a kill switch, so we should completely regulate them, stop their development, and forbid them. Everything should go through Anthropic for the sake of humanity because they have a red-button kill switch.

This doom hype is becoming ridiculous.

Re: AI 'kill switch' may need to be mandatory, Anthropic co-founder tells BBC

#55

Post by vouaobrasil »

SketchySeaBeast wrote: post #39

This feels like doom hype. These LLMs aren't Ultron, they aren't going to disseminate onto the net and hide in a smart toaster. We know where they are, in the giant facilities that draw more power than a small city and whose water consumption can be compared to golf courses, but it does make them sound all the more cool and powerful if we suggest we need a dramatic kill switch to be able to stop them in case of rampa…

LLMs don't need to hide anywhere to be dangerous, though. The danger can come purely from them reaching sufficient power together with some other idiot sufficiently crazy to use them to cause destruction.

I mean, in another thread somewhere around here, someone built an entire OS with an AI. What's to stop anyone with a sufficient taste for power to eventually use AI to cause havoc with it?

I think the underlying assumption in your post, which I believe is false, is that people are united somehow against catastrophe. They're not. There are plenty of people who participate in society currently but who would be more than happy to eradicate us normal people under different circumstances. Society often seems stable but it's far more fragile than we think.

Re: AI 'kill switch' may need to be mandatory, Anthropic co-founder tells BBC

#56

Post by foobar1274278 »

Dwar Ev ceremoniously soldered the final connection with gold. The eyes of a dozen television cameras watched him and the sub-ether bore through the universe a dozen pictures of what he was doing.

He straightened and nodded to Dwar Reyn, then moved to a position beside the switch that would complete the contact when he threw it. The switch that would connect, all at once, all of the monster computing machines of all the populated planets in the universe – ninety-six billion planets – into the super-circuit that would connect them all into the one super-calculator, one cybernetics machine that would combine all the knowledge of all the galaxies.

Dwar Reyn spoke briefly to the watching and listening trillions. Then, after a moment’s silence, he said, “Now, Dwar Ev.”

Dwar Ev threw the switch. There was a mighty hum, the surge of power from ninety-six billion planets. Lights flashed and quieted along the miles-long panel.

Dwar Ev stepped back and drew a deep breath. “The honor of asking the first question is yours, Dwar Reyn.”

“Thank you,” said Dwar Reyn. “It shall be a question that no single cybernetics machine has been able to answer.”

He turned to face the machine. “Is there a God?”

The mighty voice answered without hesitation, without the clicking of single relay.

“Yes, now there is a God.”

Sudden fear flashed on the face of Dwar Ev. He leaped to grab the switch.

A bolt of lightning from the cloudless sky struck him down and fused the switch shut.

(Fredric Brown, "Answer". 1954)

Re: AI 'kill switch' may need to be mandatory, Anthropic co-founder tells BBC

#57

Post by monological »

They're just scared of China releasing better open weights, nipping at their heels. With so much investor cash on the line, they have to create this narrative to scare the public into forcing regulation. Why would they want to be regulated? It seems counterintuitive, but it's because they want regulatory capture.

Re: AI 'kill switch' may need to be mandatory, Anthropic co-founder tells BBC

#58

Post by Kinrany »

shafyy wrote: post #15

The boy who cried wolf but the wolf never comes

The analogy breaks down when the wolf in question may very well eat the whole village: even if the boys who cry wolf are right half the time, every surviving village would have a history of no wolf ever coming to eat them

Re: AI 'kill switch' may need to be mandatory, Anthropic co-founder tells BBC

#60

Post by Barbing »

danieltk76 wrote: post #2

Why was this not a thing BEFORE continuing to develop AI? Makes me think that if they actually believed in AI causing extinction, they would have already had a kill switch.

Related, on pacing:

> Slowing down in order to address their alignment risks felt like trying to study the psychology of humans by performing experiments on bacteria.

Note author’s small financial ties to the subject (Anthropic CEO) https://darioamodei.com/post/we-must-pace-the-frontier

Post reply on HN