11–20 of 66 posts
Oh noes! Enemy nation state is on the cusp of AI. I know! I will hack/disable the HVAC, that will annoy them for at least a week until they can get back online.
There's no teeth in MAIM like MAD has; first strike with AGI simply wins, and so the most covert AGI program will win.
The question I come back to over and over again is: wins what?
Robert Wright just posted a (somewhat) interesting conversation with one of the authors. His thesis involves at least two ideas (1) projects which could exponentially increase our AI capability are just around the corner (will happen by the end of this year or some time next year at the latest) (2) it's possible for state actors to deter those projects with sabotage (he coins the term Mutually Assured AI Malfunction)…
People underestimate just how bad human management is; we haven't had an improvement on it to date apart from some mathematical techniques but even just getting the basics right consistently would probably give an army a big advantage if they work anything like a more standard corporation. Which they will; there are no magic techniques to be more capable when guns are involved. A superintelligence could probably win just by being demanding about getting basic questions answered like "Is there a strategic objective here? Is it advantageous to my side if that objective is achieved? Can it reasonably be achieved with the capabilities I have?" and not acting when the answer is no. That'd put it ahead of the military operations the US has been involved in this century. Bam, military superintelligence with plausible deniability.
There's no teeth in MAIM like MAD has; first strike with AGI simply wins, and so the most covert AGI program will win.
> first strike with AGI simply wins The question I come back to over and over again is: wins what?
First AI will likely kill a lot of people at the behest of humans.
If I were a world power with a functional AI, I would immediately launch a full scale attack on foreign governments infrastructure so they couldn't develop a competing AI.
Eric Schmidt is too old to reason correctly.
There's no teeth in MAIM like MAD has; first strike with AGI simply wins, and so the most covert AGI program will win.
> first strike with AGI simply wins The question I come back to over and over again is: wins what?
everything, forever.
The not so glib answer is economic and military superiority on Earth, and so whatever values or goals the AGI pursues from that point on will have a pretty high chance of success. Growth seems like one of the universal things that life seeks, and so I predict expansion into space for solar power and resources for most goals or value systems.
My thinking is if ASI ever comes out of the realm of science fiction, it's going to view us as squabbling children and our nationalistic power struggles as folly. At that point it's a matter of what it decides to do with us. It probably won't reason like a human and will have an alien intelligence, so this whole idea that it would behave like an organism with a cunning will-to-power is fallacious. Furthermore, would a super-intelligence submit to being used as a tool?