Live data from Hacker News

Why So Many AI Researchers Think the Machines Could Kill Everyone

wired.com

21–30 of 39 posts

Re: Why So Many AI Researchers Think the Machines Could Kill Everyone

#21

Right now there's pretty limited set of actions that rogue AI can do with internet access. The most dangerous options are hacking into biolab facility, and maybe hacking into nuclear reactors or nuclear weapons. These things should be as secure as possible, and I don't think rogue AIs, even super intelligent ones, have the capability to destroy humanity. Not to mention, super intelligent AIs would know that destroyin…

As mentioned by another commenter, the real danger is that these tools act as amplifiers and enablers for human agents that want to do really bad things. Whereas some crazy group that wants to destroy X for some reason, but had no ability to do so before, with the help of advanced AI tools might have a chance.

The other is the human deploying these things don't even need to be bad (evil) actors. They can just be really dumb and/or lazy in how they use them and then inadvertently cause huge amounts of real damage.

As Margaret Atwood pointed out "Stupidity is the same as evil if you judge by the results"

Re: Why So Many AI Researchers Think the Machines Could Kill Everyone

#22

Right now there's pretty limited set of actions that rogue AI can do with internet access. The most dangerous options are hacking into biolab facility, and maybe hacking into nuclear reactors or nuclear weapons. These things should be as secure as possible, and I don't think rogue AIs, even super intelligent ones, have the capability to destroy humanity. Not to mention, super intelligent AIs would know that destroyin…

[deleted]

Re: Why So Many AI Researchers Think the Machines Could Kill Everyone

#23

From TFA: "AI hardly needs to wipe out humanity in order to be harmful, though. Many experts predict that more powerful models will lead to a coming wave of AI-assisted cyberattacks. The technology is now widely used for disinformation campaigns, and military adoption of AI is accelerating rapidly." I fear the vast (and growing) human incompetence in essential domains. Education and science are crucial to society, an…

yeah, what's they're describing is essentially: look how morally unopposed and devoid corporations are and we're just letting them develop AI without any restraints on both capitalism and the power resultant from AI.

> the power resultant from AI

Given sufficient human incompetence, the technology doesn't even need to be powerful or efficient (or even work as advertised). It just needs to amplify/scale human incompetence.

Re: Why So Many AI Researchers Think the Machines Could Kill Everyone

#24

Narrative collapse. A common retort to someone talking about extinction risk is "only people which don't work on AI research or understand transformers are scared. people that work on LLMs know better than to fear matrix multiplication".

That's a really weird argument. That's like saying that we shouldn't be afraid of radiation because it's just decaying atoms

Re: Why So Many AI Researchers Think the Machines Could Kill Everyone

#25

Right now there's pretty limited set of actions that rogue AI can do with internet access. The most dangerous options are hacking into biolab facility, and maybe hacking into nuclear reactors or nuclear weapons. These things should be as secure as possible, and I don't think rogue AIs, even super intelligent ones, have the capability to destroy humanity. Not to mention, super intelligent AIs would know that destroyin…

As mentioned by another commenter, the real danger is that these tools act as amplifiers and enablers for human agents that want to do really bad things. Whereas some crazy group that wants to destroy X for some reason, but had no ability to do so before, with the help of advanced AI tools might have a chance. The other is the human deploying these things don't even need to be bad (evil) actors. They can just be real…

This is the same description as any other technology including fire.

Re: Why So Many AI Researchers Think the Machines Could Kill Everyone

#26
Why? That's easy.

They believe the sci-fi, rather than the reality. They've bought their own hype, and abandoned their critical thinking skills.

They're the guy saying to the computer "Say you're alive", and then being astonished when the computer says "I am alive."

Re: Why So Many AI Researchers Think the Machines Could Kill Everyone

#27
post #26

Why? That's easy. They believe the sci-fi, rather than the reality. They've bought their own hype, and abandoned their critical thinking skills. They're the guy saying to the computer "Say you're alive", and then being astonished when the computer says "I am alive."

[dead]

Re: Why So Many AI Researchers Think the Machines Could Kill Everyone

#28
post #6

It amazes me how pieces like this never stop to question whether these beliefs are genuine. If I'm one of these researchers who's leaving their position, "I am so smart that I can make something world endingly dangerous, but also so principled that I'm willing to walk away from it" is a pretty slick bit of self promotion. Even if I was making a career change for unrelated reasons, that's absolutely the story I'd be t…

And no one wants to take action, they want others to. Anthropic CEO urges AI companies to slow model development amid fears over misuse [1]. If they truly believe that, they would announce "we are slowing model development."

https://www.reuters.com/business/anthropic-ceo-urges-ai-comp...

Re: Why So Many AI Researchers Think the Machines Could Kill Everyone

#29
My bet is that humans will kill humans faster than AI. AI enables not only good actors, but also bad actors: Russians, Iranians, Houthis, other terrorist groups - they all now effectively use AI to speed-up development of their weapons as well as hybrid warfare.

Re: Why So Many AI Researchers Think the Machines Could Kill Everyone

#30
post #29

My bet is that humans will kill humans faster than AI. AI enables not only good actors, but also bad actors: Russians, Iranians, Houthis, other terrorist groups - they all now effectively use AI to speed-up development of their weapons as well as hybrid warfare.

You misspelled Israel and project nimbus
Post reply on HN