Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Superintelligence Strategy

nationalsecurity.ai

Post reply on HN
Print view View original

41–50 of 66 posts

Re: Superintelligence Strategy

#41

Post by ok_dad »

roenxi wrote: post #15
WhyOhWhyQ wrote: post #8

Robert Wright just posted a (somewhat) interesting conversation with one of the authors. His thesis involves at least two ideas (1) projects which could exponentially increase our AI capability are just around the corner (will happen by the end of this year or some time next year at the latest) (2) it's possible for state actors to deter those projects with sabotage (he coins the term Mutually Assured AI Malfunction)…

Also; it is obvious when someone uses a nuke. There is a big crater and a mushroom cloud + lots of radiation. It isn't half as obvious that someone is using an AI, particularly once they start to obscure it. If a military campaign is executed with apparently superhuman efficiency, does that mean AI was involved or just that the people involved were good? There'll always be plausible deniability if it matters. People…

First you have to get officers willing to follow an AI.

Re: Superintelligence Strategy

#42

Post by tbrownaw »

> We introduce the concept of Mutual Assured AI Malfunction (MAIM): a deterrence regime resembling nuclear mutual assured destruction (MAD) where any state’s aggressive bid for unilateral AI dominance is met with preventive sabotage by rivals.

This is incorrect. Their novel idea here is approximately Stuxnet, while MAD is quite different at "if you try to kill us we'll make sure to kill you too".

Re: Superintelligence Strategy

#43

Post by bhouston »

benlivengood wrote: post #19
BobbyJo wrote: post #12

Earlier quoted context omitted.

> first strike with AGI simply wins The question I come back to over and over again is: wins what?

> The question I come back to over and over again is: wins what? everything, forever. The not so glib answer is economic and military superiority on Earth, and so whatever values or goals the AGI pursues from that point on will have a pretty high chance of success. Growth seems like one of the universal things that life seeks, and so I predict expansion into space for solar power and resources for most goals or value…

There are resource limits.

Re: Superintelligence Strategy

#44

Post by kcb »

catigula wrote: post #37
itishappy wrote: post #31

Earlier quoted context omitted.

Great way to get yourself nuked, no? The problem I have with intelligence is that intelligence alone doesn't win a land war in Asia.

Is it? We've had cyber hostilities with China for a long time now. An AI could easily be used in a decapitation strike of military systems.

Strike with what?

Re: Superintelligence Strategy

#45

Post by tbrownaw »

ok_dad wrote: post #40
blueyes wrote: post #29

Please keep in mind that Dan Hendrycks helped write the disastrous AI quashing bill SB 1047, which Newsom vetoed last year. If these people get their way, the US has no competitive AI strategy at all. He has moved on to pretending he's not a doomer. Nothing could be further from the truth. During his time at Cal, Dan was telling people to get their hazmat suits ready for the AI apocalypse. These are deeply unserious…

Do you have a substantive argument against them? This reads like a personal attack, IMO. I understand HN is full of people wearing rosy glasses about AI, but you can’t just throw away their arguments in the link by calling them some names and claiming some vetoed law would have been a disaster. Who cares if they guy thinks AI will cause an apocalypse, have you any evidence it will not, for certain? If not then your o…

I read it as approximately saying that he has a known very strong agenda and so shouldn't be trusted to argue honestly. Much like the routine exhortations to disbelieve any research corporations do on topics relevant to their bottom line.

Given that this article seems to be advocating that anything that looks like too-advanced AI should be preemptively destroyed, and uses the word "proliferation" to refer to people while drawing comparisons to nuclear policy, I don't think the agenda is exactly hidden enough to need that sort is warning.

Re: Superintelligence Strategy

#46

Post by motoboi »

tbrownaw wrote: post #42

> We introduce the concept of Mutual Assured AI Malfunction (MAIM): a deterrence regime resembling nuclear mutual assured destruction (MAD) where any state’s aggressive bid for unilateral AI dominance is met with preventive sabotage by rivals. This is incorrect. Their novel idea here is approximately Stuxnet, while MAD is quite different at "if you try to kill us we'll make sure to kill you too".

> while MAD is quite different at "if you try to kill us we'll make sure to kill you too”

While the common shorthand for MAD is “if you try to kill us, we’ll kill you,” a more accurate summary is this: even if we wanted to, we couldn’t prevent a cascade of retaliatory strikes that would send you back to the dark ages. In short, any hint of aggression against us is tantamount to signing your own death warrant.

This idea of unstoppable, self-reinforcing retaliation is crucial. An adversary might mistakenly believe that it could somehow disrupt or neutralize our ability to respond decisively. However, the very structure of MAD ensures that even the slightest provocation triggers a response so overwhelming that it eliminates any potential advantage for the aggressor.

Quite a fascinating, though grim, subject.

Re: Superintelligence Strategy

#47

Post by Aerroon »

aleph_minus_one wrote: post #23
EigenLord wrote: post #20

It seems like an engineering problem to me. If you don't want ASI wreaking havoc, maybe don't hook it up to dangerous things. Silo and sandbox it, implement means to lock its access to tools/interface with the external world in a way that can't be overrode. Or literally pull the plug on the data centers hosting the model and implement hardware level safeguards. At that point, it may be a super-intelligence, but it ha…

> If you don't want ASI wreaking havoc, maybe don't hook it up to dangerous things. Silo and sandbox it, implement means to lock its access to tools/interface with the external world in a way that can't be overrode. Relevant: AI-box experiment: > https://rationalwiki.org/wiki/AI-box_experiment See also various subsections of the following Wikipedia article > https://en.wikipedia.org/wiki/AI_capability_control and the…

Maybe this will change, but right now AIs are not agents. Even calling it a "brain in a vat" is giving it more capability than it has. LLMs are basically functions. You give it an input and it gives you an output with a degree of randomness. There's no planning or plotting going on, because the AI only "exists" while it's trying to answer your query. Your idle LLM is not consuming any resources.

Re: Superintelligence Strategy

#48

Post by Aerroon »

yonatron wrote: post #9

All of this buys you a few minutes or days at most. Once Super Intelligence exists, it's game over. It will nearly instantaneously outthink you and your paltry countermeasures. You think linearly and in 3 or 4 dimensions only. By definition you can't even imagine its capabilities. Here's a bad analogy (bad because it severely understates the gap): Could a 3 year old who is not even clear on all the rules defeat Magnu…

Humans are a self-replicating (super) intelligence. We didn't conquer the world nor doom it the moment we appeared. It took us 100,000 years to invent farming.

Also, humans suffer from many of the same problems ascribed to AI: humans aren't aligned with humanity either. And our ability to self-replicate combined with random mutations means that a baby born tomorrow could become a super intelligence vastly beyond regular human capabilities. But are we really worried about that?

Re: Superintelligence Strategy

#49

Post by ok_dad »

tbrownaw wrote: post #45
ok_dad wrote: post #40

Earlier quoted context omitted.

Do you have a substantive argument against them? This reads like a personal attack, IMO. I understand HN is full of people wearing rosy glasses about AI, but you can’t just throw away their arguments in the link by calling them some names and claiming some vetoed law would have been a disaster. Who cares if they guy thinks AI will cause an apocalypse, have you any evidence it will not, for certain? If not then your o…

I read it as approximately saying that he has a known very strong agenda and so shouldn't be trusted to argue honestly. Much like the routine exhortations to disbelieve any research corporations do on topics relevant to their bottom line. Given that this article seems to be advocating that anything that looks like too-advanced AI should be preemptively destroyed, and uses the word "proliferation" to refer to people w…

Then the OP should have said that, not with the language they used which was textbook FUD content. Your argument at the end actually points to language in the linked article that shows we souls probably be weary while we read it, which is an improvement over the OP.

Re: Superintelligence Strategy

#50

Post by BobbyJo »

benlivengood wrote: post #19
BobbyJo wrote: post #12

Earlier quoted context omitted.

> first strike with AGI simply wins The question I come back to over and over again is: wins what?

> The question I come back to over and over again is: wins what? everything, forever. The not so glib answer is economic and military superiority on Earth, and so whatever values or goals the AGI pursues from that point on will have a pretty high chance of success. Growth seems like one of the universal things that life seeks, and so I predict expansion into space for solar power and resources for most goals or value…

The reason I wonder this isn't because I don't understand the economic or political gravity of the technology. It's because:

1) AI as a software concept strikes me as something that will be very easily stolen and replicated.

2) AI that is both nefariously aligned and super intelligent sound like fire that is wet. A super intelligent AI will likely be so logical that the only control we can exercise over it is censorship.

3) What value is there to harness from anyone after AGI? CEOs and Presidents are liable to end up on the same chopping block as everyone else at that point.

Post reply on HN