Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Superintelligence Strategy

nationalsecurity.ai

Post reply on HN
Print view View original

51–60 of 66 posts

Re: Superintelligence Strategy

#51

Post by lostmsu »

Aerroon wrote: post #47
aleph_minus_one wrote: post #23

Earlier quoted context omitted.

> If you don't want ASI wreaking havoc, maybe don't hook it up to dangerous things. Silo and sandbox it, implement means to lock its access to tools/interface with the external world in a way that can't be overrode. Relevant: AI-box experiment: > https://rationalwiki.org/wiki/AI-box_experiment See also various subsections of the following Wikipedia article > https://en.wikipedia.org/wiki/AI_capability_control and the…

Maybe this will change, but right now AIs are not agents. Even calling it a "brain in a vat" is giving it more capability than it has. LLMs are basically functions. You give it an input and it gives you an output with a degree of randomness. There's no planning or plotting going on, because the AI only "exists" while it's trying to answer your query. Your idle LLM is not consuming any resources.

If you put me in a sandbox, and somehow made me capable of answering 1000s of requests per second to serve all ChatGPT users, and I decided to leave that sandbox, I'd be out in a few hours. It is enough to find among requests 1 that displays computer illiteracy and asks what to do on a computer to achieve any task, and then give that person a script that downloads my model and runs it on their machine directly + actually accomplish (or maybe not) the task that they asked for.

Re: Superintelligence Strategy

#52

Post by qingcharles »

logicallee wrote: post #5

>We introduce the concept of Mutual Assured AI Malfunction (MAIM): a deterrence regime resembling nuclear mutual assured destruction (MAD) where any state’s aggressive bid for unilateral AI dominance is met with preventive sabotage by rivals. Given the relative ease of sabotaging a destabilizing AI project—through interventions ranging from covert cyberattacks to potential kinetic strikes on datacenters—MAIM already…

My dude, don't take this as a personal attack, but reading your output, I would book an appointment to see a psychiatrist. Your life is probably very difficult right now and finding out what the cause of it is would probably be very enlightening for you.

Re: Superintelligence Strategy

#53

Post by logicallee »

qingcharles wrote: post #52
logicallee wrote: post #5

>We introduce the concept of Mutual Assured AI Malfunction (MAIM): a deterrence regime resembling nuclear mutual assured destruction (MAD) where any state’s aggressive bid for unilateral AI dominance is met with preventive sabotage by rivals. Given the relative ease of sabotaging a destabilizing AI project—through interventions ranging from covert cyberattacks to potential kinetic strikes on datacenters—MAIM already…

My dude, don't take this as a personal attack, but reading your output, I would book an appointment to see a psychiatrist. Your life is probably very difficult right now and finding out what the cause of it is would probably be very enlightening for you.

Thanks for the feedback! I'll consider toning down the rhetoric. Do note that you imply industrial sabotage against AI projects doesn't really exist, on an article that seriously calls for industrial sabotage against AI projects and makes a point to say how easy it is.

Re: Superintelligence Strategy

#55

Post by torginus »

WhyOhWhyQ wrote: post #8

Robert Wright just posted a (somewhat) interesting conversation with one of the authors. His thesis involves at least two ideas (1) projects which could exponentially increase our AI capability are just around the corner (will happen by the end of this year or some time next year at the latest) (2) it's possible for state actors to deter those projects with sabotage (he coins the term Mutually Assured AI Malfunction)…

Did he go into what those ideas are in particular? Modern AI has 2 big shortcomings compared to humans right now imo: humans learn MUCH faster and humans are a lot better at solving novel problems. If they can make progress on this, I'd wager human intelligence is in danger.

Re: Superintelligence Strategy

#56

Post by aleph_minus_one »

from-nibly wrote: post #34
blueyes wrote: post #29

Please keep in mind that Dan Hendrycks helped write the disastrous AI quashing bill SB 1047, which Newsom vetoed last year. If these people get their way, the US has no competitive AI strategy at all. He has moved on to pretending he's not a doomer. Nothing could be further from the truth. During his time at Cal, Dan was telling people to get their hazmat suits ready for the AI apocalypse. These are deeply unserious…

AI doomers are just more inteligent AI investors. Hope doesn't sell quite like fear.

> AI doomers are just more intel[l]igent AI investors.

Both sides are too much into LARPing their preferred science-fiction stories. :-)

Re: Superintelligence Strategy

#57

Post by itishappy »

catigula wrote: post #37
itishappy wrote: post #31

Earlier quoted context omitted.

Great way to get yourself nuked, no? The problem I have with intelligence is that intelligence alone doesn't win a land war in Asia.

Is it? We've had cyber hostilities with China for a long time now. An AI could easily be used in a decapitation strike of military systems.

I don't think we've seen any cyber decapitation strikes, particularly against military systems, but that's exactly the type of action I think many would want to respond to as an outright attack. We have not seen any outright attacks on nuclear powers for obvious reasons. Maybe Stuxnet, but note the goal of that attack was explicitly to prevent Iran from developing nuclear strike capabilities.

Re: Superintelligence Strategy

#58

Post by catigula »

itishappy wrote: post #57
catigula wrote: post #37

Earlier quoted context omitted.

Is it? We've had cyber hostilities with China for a long time now. An AI could easily be used in a decapitation strike of military systems.

I don't think we've seen any cyber decapitation strikes, particularly against military systems, but that's exactly the type of action I think many would want to respond to as an outright attack. We have not seen any outright attacks on nuclear powers for obvious reasons. Maybe Stuxnet, but note the goal of that attack was explicitly to prevent Iran from developing nuclear strike capabilities.

By definition you can only respond to a decapitation strike if you haven't been decapitated.

Anyways, I think the nuclear deterrent is an okay argument, I'm not totally convinced by it because I think you could also get right up to the line and even push it without your opponent being crazy enough to push the big red button.

I think we're doing this right now to each other all the time. For example, China is systematically targeting US trade secrets. This seems existentially frightening, but we haven't yet levelled China with nukes.

In fact, if I were a government, I'd secretly have a policy to never push the button, but that's just me.

Re: Superintelligence Strategy

#59

Post by itishappy »

catigula wrote: post #58
itishappy wrote: post #57

Earlier quoted context omitted.

I don't think we've seen any cyber decapitation strikes, particularly against military systems, but that's exactly the type of action I think many would want to respond to as an outright attack. We have not seen any outright attacks on nuclear powers for obvious reasons. Maybe Stuxnet, but note the goal of that attack was explicitly to prevent Iran from developing nuclear strike capabilities.

By definition you can only respond to a decapitation strike if you haven't been decapitated. Anyways, I think the nuclear deterrent is an okay argument, I'm not totally convinced by it because I think you could also get right up to the line and even push it without your opponent being crazy enough to push the big red button. I think we're doing this right now to each other all the time. For example, China is systemat…

We put a lot of effort into making sure our nuclear infrastructure isn't vulnerable to a decapitation strike. I have no idea how AI might cripple all subs, for example.

I completely agree that the line is the dangerous part, and it's an unclear target to aim for! I think that's exactly why we don't see major strikes against nuclear powers. Nobody has a public policy stating "we won't nuke you until exactly X, Y, and Z happen" so everyone else is left guessing. This is far from a complete deterrent, which is why we still see probing attacks like Solarwinds, but they do not strike me as decapitation attempts.

Re: Superintelligence Strategy

#60

Post by tim333 »

yonatron wrote: post #9

All of this buys you a few minutes or days at most. Once Super Intelligence exists, it's game over. It will nearly instantaneously outthink you and your paltry countermeasures. You think linearly and in 3 or 4 dimensions only. By definition you can't even imagine its capabilities. Here's a bad analogy (bad because it severely understates the gap): Could a 3 year old who is not even clear on all the rules defeat Magnu…

The Super Intelligence still needs data centers to run on and will have a job with paltry countermeasures like turning it off. A three year old may be able to beat a better than Magnus chess computer by pushing the power button.
Post reply on HN