Earlier quoted context omitted.
> The nonexistence of grey goo (von neumann probes) is strong prior for safe agi. what if we're first? or FTL travel isn't possible in our universe?
Sure, have a different prior if you think those assumptions are relevant. i don't P(what if we're first?) ~ 0 P(or FTL travel isn't possible in our universe?) ~ 1
AGI Doom and the Drake Equation
111–113 of 113 posts
Re: AGI Doom and the Drake Equation
#112As usual with groundbreaking technology, reckless use by military/intelligence agencies are the greatest consequential threat. If we rank unbounded AI on the same potential threat level as nuclear, chemical and biological warfare, than history says that's who will be the first to utilize it in the worst ways possible. I do notice the federal government is going all-in on AI contracts at present, here's the non-black-…
There are people talking about it obviously.
> That's a pretty solid argument for continued public access to these tools, on top of the incredible educational potentials.
We have millions of books yet many people haven't even read the Bible they nominally base their uneducated life on. :|
Re: AGI Doom and the Drake Equation
#113Something I do not see represented in these arguments: real world conditions. Most complex computer systems (which we assume to be the case for a super powerful AI) don't run for very long without requiring manual intervention of some sort. "Aha!" I hear you saying, "The AI will figure out how to reboot nodes and scale clusters and such". OK fine. But then there is the meat space... Replacing hardware, running power…
Solving those are simple tasks of manufacturing power plants, water treatment plants, fertilizer plants, agricultural machines, logistics of deploying them and keeping them fueled, and persuading people to take the fucking vaccine, use PrEP and use condoms. Money solves these. There's enough people and resources to do this.
The AGI foom argument basically says that the AGI will commandeer the economy of the size of a developed country, well, let's say Germany. (Let's pick Germany, because we already saw what that country was able to do when run by a dictator.)
Insisting that AGI can organize enough people to replace enough broken hardware while manufacturing new hardware doesn't seem that far fetched. (Again, Germany was able to increase its industrial production during WWII while the Allies bombed it constantly for months.)