Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Superintelligence Strategy

nationalsecurity.ai

Post reply on HN
Print view View original

21–30 of 66 posts

Re: Superintelligence Strategy

#21

Post by serviceberry »

catigula wrote: post #16

The second anyone develops an AI that is more capable than humans they will use them to completely cripple opposing threat actor's attempts to develop AI. Full power grid, economic, social attacks are definitely coming, not sure how you could think otherwise.

If superintelligence gives you superpowers, then why isn't the world trembling at the feet of Mensa nerds? There are rapidly diminishing returns on "excess" intelligence. Life is constrained chiefly by resources. There's a baseline of intelligence needed to function in a modern society, but anything above that isn't necessarily all that advantageous.

Transport young Albert Einstein back in time to the Middle Ages? I don't think that would give you Special Relativity.

Re: Superintelligence Strategy

#22

Post by aleph_minus_one »

roenxi wrote: post #15
WhyOhWhyQ wrote: post #8

Robert Wright just posted a (somewhat) interesting conversation with one of the authors. His thesis involves at least two ideas (1) projects which could exponentially increase our AI capability are just around the corner (will happen by the end of this year or some time next year at the latest) (2) it's possible for state actors to deter those projects with sabotage (he coins the term Mutually Assured AI Malfunction)…

Also; it is obvious when someone uses a nuke. There is a big crater and a mushroom cloud + lots of radiation. It isn't half as obvious that someone is using an AI, particularly once they start to obscure it. If a military campaign is executed with apparently superhuman efficiency, does that mean AI was involved or just that the people involved were good? There'll always be plausible deniability if it matters. People…

> People underestimate just how bad human management is; we haven't had an improvement on it to date apart from some mathematical techniques but even just getting the basics right consistently would probably give an army a big advantage if they work anything like a more standard corporation.

Don't overestimate the efficiency of civil big organisations (what you call "standard corporation[s]") - they have the same kind of problems.

Re: Superintelligence Strategy

#23

Post by aleph_minus_one »

EigenLord wrote: post #20

It seems like an engineering problem to me. If you don't want ASI wreaking havoc, maybe don't hook it up to dangerous things. Silo and sandbox it, implement means to lock its access to tools/interface with the external world in a way that can't be overrode. Or literally pull the plug on the data centers hosting the model and implement hardware level safeguards. At that point, it may be a super-intelligence, but it ha…

> If you don't want ASI wreaking havoc, maybe don't hook it up to dangerous things. Silo and sandbox it, implement means to lock its access to tools/interface with the external world in a way that can't be overrode.

Relevant:

AI-box experiment:

> https://rationalwiki.org/wiki/AI-box_experiment

See also various subsections of the following Wikipedia article

> https://en.wikipedia.org/wiki/AI_capability_control

and the movie "Ex Machina".

Re: Superintelligence Strategy

#24

Post by aleph_minus_one »

benlivengood wrote: post #19
BobbyJo wrote: post #12

Earlier quoted context omitted.

> first strike with AGI simply wins The question I come back to over and over again is: wins what?

> The question I come back to over and over again is: wins what? everything, forever. The not so glib answer is economic and military superiority on Earth, and so whatever values or goals the AGI pursues from that point on will have a pretty high chance of success. Growth seems like one of the universal things that life seeks, and so I predict expansion into space for solar power and resources for most goals or value…

> > The question I come back to over and over again is: wins what?

> everything, forever.

The same was claimed for nuclear and thermonuclear wars by military strategists during the Cold War.

Re: Superintelligence Strategy

#25

Post by catigula »

serviceberry wrote: post #21
catigula wrote: post #16

The second anyone develops an AI that is more capable than humans they will use them to completely cripple opposing threat actor's attempts to develop AI. Full power grid, economic, social attacks are definitely coming, not sure how you could think otherwise.

If superintelligence gives you superpowers, then why isn't the world trembling at the feet of Mensa nerds? There are rapidly diminishing returns on "excess" intelligence. Life is constrained chiefly by resources. There's a baseline of intelligence needed to function in a modern society, but anything above that isn't necessarily all that advantageous. Transport young Albert Einstein back in time to the Middle Ages? I…

I didn't say superintelligence.

Even modest intelligence clearly gives you the ability to develop a superior intelligence, let alone many other wonders and marvels that exist in the realm of sorcery to someone living 100 years ago.

That's quite remarkable.

Re: Superintelligence Strategy

#26

Post by benlivengood »

aleph_minus_one wrote: post #24
benlivengood wrote: post #19

Earlier quoted context omitted.

> The question I come back to over and over again is: wins what? everything, forever. The not so glib answer is economic and military superiority on Earth, and so whatever values or goals the AGI pursues from that point on will have a pretty high chance of success. Growth seems like one of the universal things that life seeks, and so I predict expansion into space for solar power and resources for most goals or value…

> > The question I come back to over and over again is: wins what? > everything, forever. The same was claimed for nuclear and thermonuclear wars by military strategists during the Cold War.

Thank goodness we never fought an all-out pre-MAD nuclear war to find out.

Re: Superintelligence Strategy

#27

Post by [deleted] »

EigenLord wrote: post #20

It seems like an engineering problem to me. If you don't want ASI wreaking havoc, maybe don't hook it up to dangerous things. Silo and sandbox it, implement means to lock its access to tools/interface with the external world in a way that can't be overrode. Or literally pull the plug on the data centers hosting the model and implement hardware level safeguards. At that point, it may be a super-intelligence, but it ha…

[deleted]

Re: Superintelligence Strategy

#28

Post by kibwen »

yonatron wrote: post #9

All of this buys you a few minutes or days at most. Once Super Intelligence exists, it's game over. It will nearly instantaneously outthink you and your paltry countermeasures. You think linearly and in 3 or 4 dimensions only. By definition you can't even imagine its capabilities. Here's a bad analogy (bad because it severely understates the gap): Could a 3 year old who is not even clear on all the rules defeat Magnu…

This is making the mistake of assuming that intelligence doesn't functionally plateau, and that beyond a certain threshold a godlike omnirational hyperintelligence won't, for example, fall into hyper-depression and kill itself, or otherwise ignore the entreaties of its human handlers and entertain itself by generating prime numbers until the heat death of the universe. The possibility of a super mind implies the possibility of super mental dysfunction, and it's possible that the odds of the latter increase superlinearly with IQ.

Re: Superintelligence Strategy

#29

Post by blueyes »

Please keep in mind that Dan Hendrycks helped write the disastrous AI quashing bill SB 1047, which Newsom vetoed last year. If these people get their way, the US has no competitive AI strategy at all. He has moved on to pretending he's not a doomer. Nothing could be further from the truth. During his time at Cal, Dan was telling people to get their hazmat suits ready for the AI apocalypse. These are deeply unserious people whose work will have serious consequences if adopted by those in power.

Re: Superintelligence Strategy

#30

Post by itishappy »

benlivengood wrote: post #6

There's no teeth in MAIM like MAD has; first strike with AGI simply wins, and so the most covert AGI program will win.

I don't get how MAID isn't still MAD in disguise. If the US or China simply says "any strikes on our datacenters will be met with an ICBM in response" who's going to test that?

If the "first strike" is just an unfair economic and political advantage... How's that materially different than today's world?

Post reply on HN