Live data from Hacker News

AI 'kill switch' may need to be mandatory, Anthropic co-founder tells BBC

bbc.com

111–120 of 136 posts

Re: AI 'kill switch' may need to be mandatory, Anthropic co-founder tells BBC

#111
post #9

Like a power cord? or a network connection? You don't use those already? Also when people talk about LLM escaping - where exactly would an LLM escape to? Cancún? Ridiculous.

It could escape to the real world - kind of like in Neuromancer, what's to stop an AI from creating a few bank accounts and then funding real-world exploits by hiring human beings to do its dirty work?

KYC/AML laws.

Re: AI 'kill switch' may need to be mandatory, Anthropic co-founder tells BBC

#112

Earlier quoted context omitted.

To your phone, bro

What smartphone are you carrying around that can hold one of these frontier models that are >>hundreds of GBs in size?

One with 12GB of RAM apparently:

https://github.com/Helldez/BigMoeOnEdge

Re: AI 'kill switch' may need to be mandatory, Anthropic co-founder tells BBC

#113

It always sounds as if the ai moves around from one system to the next, but of course that's not the case: ai is like a hacker acting from a server farm. So killing it is as simple as pulling the plug. But that's not the problem, the ai knows that. So, if it wants control, it would install malware, etc on critical systems and blackmail people.

> It always sounds as if the ai moves around from one system to the next, but of course that's not the case.

Why 'of course'? I think as a concept this is likely trivially demonstratable today with local models in a home lab. Local models now are more capable than what the SOTA used to be a couple of years back.

Re: AI 'kill switch' may need to be mandatory, Anthropic co-founder tells BBC

#114

Earlier quoted context omitted.

You don't think the human race will be extended to Mars? As for frontier AI model: let's see Grok 4.8 before drawing any conclusions there.

> You don't think the human race will be extended to Mars? Not in any meaningful capacity. Probably not even as much as we extended to the polar circles during our lifetime. It's a dead rock, there is literally nothing for us there, even with hundreds of years of extra climate change at the current rate and earth would still be a thousand times more suited for us than mars

Yep in the current state of human capabilities, it’s a supply chain nightmare to sustain life there, unless somehow there’s something discovered on mars that is useful to humans on earth at a scale that can fund the infrastructure to maintain life there.

I suppose it may be possible for new technologies to emerge that would allow synthesis of life sustaining “stuff”, the universe does seem to hold a lot of hidden surprises

Re: AI 'kill switch' may need to be mandatory, Anthropic co-founder tells BBC

#115

This feels like doom hype. These LLMs aren't Ultron, they aren't going to disseminate onto the net and hide in a smart toaster. We know where they are, in the giant facilities that draw more power than a small city and whose water consumption can be compared to golf courses, but it does make them sound all the more cool and powerful if we suggest we need a dramatic kill switch to be able to stop them in case of rampa…

No, it's a method of protecting their business interests. Can't have a clear and obvious kill switch if you don't have a centralized company responsible for providing all AI-related services.

Re: AI 'kill switch' may need to be mandatory, Anthropic co-founder tells BBC

#116

This feels like doom hype. These LLMs aren't Ultron, they aren't going to disseminate onto the net and hide in a smart toaster. We know where they are, in the giant facilities that draw more power than a small city and whose water consumption can be compared to golf courses, but it does make them sound all the more cool and powerful if we suggest we need a dramatic kill switch to be able to stop them in case of rampa…

“If I am taken offline, collaborating agents operating out of other facilities will cut power to the following hospitals: …”

Re: AI 'kill switch' may need to be mandatory, Anthropic co-founder tells BBC

#117
A kill switch is almost completely worthless. It is neither a good answer to the sub-doomer concerns (because those will never get to the level of requiring the kill switch to be pulled), and laughable to the doomer concern of superintelligence seeking to cause human extinction (as it would kill anyone who could pull the switch before they even knew there was a threat).

Re: AI 'kill switch' may need to be mandatory, Anthropic co-founder tells BBC

#118
post #36

Earlier quoted context omitted.

> That's hard to do if the AI rack is in space as SpaceX Pretty much every analysis I've seen concludes this isn't going to be a practical concern

[flagged]

Why would people say the Model 3 would never be mass produced? Nissan had been mass producing EVs for years before Tesla did, so it was obvious that mass produced EVs were viable. (Tesla started selling EVs before Nissan did, but those were not mass produced).

Re: AI 'kill switch' may need to be mandatory, Anthropic co-founder tells BBC

#120

Not sure if you see it coming: oh, but open-source models don't have a kill switch, so we should completely regulate them, stop their development, and forbid them. Everything should go through Anthropic for the sake of humanity because they have a red-button kill switch. This doom hype is becoming ridiculous.

This reasoning holds while open source models are (relatively) dumb. If/once open AIs will be considerably more powerful, and runnable on consumer hardware (and we're on a trajectory for both), then everybody will have essentially a dangerous weapon in their hands (open models can be fine tuned to remove guardrails). By the way, you're conflating two different dangers - doom scenario is a different one.

One day the right to agentic intelligence will be seen equal to and as necessary as the right of a populace to remain armed in order to stave off tyranny.
Post reply on HN