Narrative collapse. A common retort to someone talking about extinction risk is "only people which don't work on AI research or understand transformers are scared. people that work on LLMs know better than to fear matrix multiplication".
[deleted]
Why So Many AI Researchers Think the Machines Could Kill Everyone
11–20 of 39 posts
Re: Why So Many AI Researchers Think the Machines Could Kill Everyone
#12Right now there's pretty limited set of actions that rogue AI can do with internet access. The most dangerous options are hacking into biolab facility, and maybe hacking into nuclear reactors or nuclear weapons.
These things should be as secure as possible, and I don't think rogue AIs, even super intelligent ones, have the capability to destroy humanity. Not to mention, super intelligent AIs would know that destroying humanity will also destroy themselves.
Maybe in 20 years once we have only self driving cars, robots in every household or maybe robots with weapons, then AI would pose a real extinction risk. But we're not there yet and it depends less on the development of AI and more on the development of physical real world robots.
Re: Why So Many AI Researchers Think the Machines Could Kill Everyone
#13It amazes me how pieces like this never stop to question whether these beliefs are genuine. If I'm one of these researchers who's leaving their position, "I am so smart that I can make something world endingly dangerous, but also so principled that I'm willing to walk away from it" is a pretty slick bit of self promotion. Even if I was making a career change for unrelated reasons, that's absolutely the story I'd be t…
[deleted]
Re: Why So Many AI Researchers Think the Machines Could Kill Everyone
#14Narrative collapse. A common retort to someone talking about extinction risk is "only people which don't work on AI research or understand transformers are scared. people that work on LLMs know better than to fear matrix multiplication".
ehhh....to be pedantic, LLMs, like any NNs derive power from _not_ being pure matrix multiplication - they are "halved" matrix algebra, due to ReLU
Re: Why So Many AI Researchers Think the Machines Could Kill Everyone
#15[dead]
Re: Why So Many AI Researchers Think the Machines Could Kill Everyone
#16AI is a problem only now where the people in power or their families are at risk of being assassinated. AI assisted drones were not a problem at all when they were sent for years to kill everybody in weddings at some obscure Asian country. None of this researchers were complaining then.
-----------------------
Is obvious that Trump fears an AI assisted killer droid attack. Change of hair tone (obviously to make any automatic face match more difficult, and confuse any long-distance shooter), thick plastic screens seen behind him at rallies to stop drones and shooters. A cavalcade of black limousines in the road with a Trump impersonator trying his best effort to be noticed... (Why would you travel in a bullet-proof vehicle with the bullet-proof window opened? the only logical explanation is trying to lure out any hitman hiding in the location).
He looks much less smug; is reducing his Chiquito de la Calzada impersonations [1], and adopting a lower profile, maybe also fearing the post-midterms. His social truth posts are filled with images of him with radioactive fire balls in his hands and traveling in planes and garbage trucks. Asking for a ride out of US, Trump?
The risk of some disruptive event in the next 60 days is greater than ever in US history. It does not need to be assisted by AI. NS (natural stupidity) is more than enough. I'm particularly concerned by the triad: Lincoln bathroom + UCF fight + distracting pool.
I don't see how this could be stopped unless every country agrees to sign treatises and follow international laws of war. Sadly this can't happen until a few old men and war criminals are jailed or die of old age. Nobody respect treatises anymore.
These men will block or delay the climate actions until is too late. Will send their army to kill left and right, will remove anti famine measures just by spite, killing thousands. They will do anything to save themselves from jail, or to earn one million more.
These men, not AI, are the real problem. But they rarely exit their bunkers nowadays. They know.
Re: Why So Many AI Researchers Think the Machines Could Kill Everyone
#17Is it because they will get funding to stop the risk even if the risk never existed in the first place to any significant extent? Are most of these wild declarations not exclusively for market value and funding?
Re: Why So Many AI Researchers Think the Machines Could Kill Everyone
#18Right now there's pretty limited set of actions that rogue AI can do with internet access. The most dangerous options are hacking into biolab facility, and maybe hacking into nuclear reactors or nuclear weapons. These things should be as secure as possible, and I don't think rogue AIs, even super intelligent ones, have the capability to destroy humanity. Not to mention, super intelligent AIs would know that destroyin…
The easy way would simply be to air gap them. The controls should be on site only.
Re: Why So Many AI Researchers Think the Machines Could Kill Everyone
#19TL;DR: Researchers assumed it would be easier to align these models when they become smarter. In reality, agents are getting increasingly smarter but aligning them also becomes increasingly harder, or even impossible. They are losing their control over the agents. They don’t write their code anymore. They don’t know agents limits nor how to limit them.
They are like lab scientists in a Hollywood thriller, watching a mutant organism rapidly evolving in front of their eyes.
Re: Why So Many AI Researchers Think the Machines Could Kill Everyone
#20From TFA: "AI hardly needs to wipe out humanity in order to be harmful, though. Many experts predict that more powerful models will lead to a coming wave of AI-assisted cyberattacks. The technology is now widely used for disinformation campaigns, and military adoption of AI is accelerating rapidly." I fear the vast (and growing) human incompetence in essential domains. Education and science are crucial to society, an…
yeah, what's they're describing is essentially: look how morally unopposed and devoid corporations are and we're just letting them develop AI without any restraints on both capitalism and the power resultant from AI.