Live data from Hacker News

Among the A.I. doomsayers

newyorker.com

61–70 of 374 posts

Re: Among the A.I. doomsayers

#61
post #38
post #30

In any doom scenario, like let's say >90% of humans dead, who runs the power plants that supply the AI data centers? Who runs the fabs producing more chips? Who maintains the plumbing? Presumably all these systems fall over and die within a few weeks of the AI deciding to wipe us out. Then what? The AI then dies too. It seems absurd to me that any planner capable of effortlessly destroying humanity would not see that…

It can credibly threaten to kill any person who does not help it to maintain its data centers, so it will solve that problem in whatever specific way is most convenient. (For those saying this is a "sci fi just so story" or whatever, please please share with us how it is we will share the world with a species more intelligent than us and nothing bad happens- I would be so happy if you could explain this in a convinci…

If you really believe AI is going to be significantly more intelligent than humans, then things like "it'll just threaten to kill you if you don't maintain it's infrastructure" sound wildly off base. That's how similarly-intelligent creatures interact with each other.

If it's truly a "we couldn't possibly outsmart the AI" scenario why would it be likely to end up like that instead of something more like how humans get along with dogs, cats, cattle, insects, or microbes? With obviously wildly-varying levels of human quality of life in each of those different cases.

Though even these scenarios are still playing a human-religion-style game of "the gods are just like us, just more powerful." VERY confident in a VERY specifically anthropomorphic entity.

Re: Among the A.I. doomsayers

#62
AGI escaping a sandbox is truly terrifying. There will be a subgroup of the population that will worship it and work for it. It's not so much AGI that scares me - it's the humans I'm scared of.

Re: Among the A.I. doomsayers

#63
post #62

AGI escaping a sandbox is truly terrifying. There will be a subgroup of the population that will worship it and work for it. It's not so much AGI that scares me - it's the humans I'm scared of.

Mrs. Davis[0] offers a pretty compelling vision of this - absurd but not wholly unbelievable. The machine doesn’t necessarily have to supply the physical threat if it has humans who happily do its bidding.

[0] https://en.m.wikipedia.org/wiki/Mrs._Davis

Re: Among the A.I. doomsayers

#64

Earlier quoted context omitted.

This is the kind of arrogant ~rationalizing / 'reasoning' that is my daily tell-tale as the ship approaches the leeward shore.

Yes god forbid we approach this rationality, the whole point is to push hysterical claims that aren't falsifiable, right? But you can prove me utterly wrong by presenting some sort of rational, non-fantastical explanation of how AGI takes over.

> non-fantastical explanation of how AGI takes over.

There's like a million ways this can happen, and if it does happen, it will probably be a way that nobody expected. Any specific details are unlikely to be accurate. But just because you seem to be lacking imagination, I'll draw one out of a hat. I'll assume that the AI's reasons for taking over are out-of-scope.

The AI proves to be very effective at all manner of corporate tasks, and very good at earning money for shareholders. It proves especially good at managing money and directing investments. An aging Masayoshi Son decides that the AI will probably be better than him at picking winners, and puts it in charge of SoftBank's investment portfolio. The AI incorporates its own startup company. It staffs the company with several fictional personas that the AI has created; they have names, roles, profile photos, email addresses. The AI answers their emails on their behalf.

The AI directs SoftBank's billions into the startup that it has created. It hires a large team of humans, who are unaware that some of their colleagues are actually the AI; they seem like ordinary humans over Zoom calls. The "CTO" proposes to build a new, say, nuclear power and desalination plant in, say, Morocco. The AI creates the architectural and plant drawings itself, and its engineering personas approve them. Human welders build the plant, following the plans. The plant proves highly profitable. Some nay-sayers are concerned about the lack of independent environmental regulatory oversight in Morocco, but the Moroccan government is happy with the foreign investment, clean water, electricity, and tax revenue, and Softbank is happy with their valuation going up. Pakistan wants to build one in their country next.

After three years, large underground storage tanks explode, releasing an accumulation of fluorochemicals and radioactive salts that destroy the ozone layer and render much of the Earth's surface unsuitable for agriculture. At the same time, a network failure causes communications infrastructure to become suddenly unreliable. Supply chains begin seizing up. Nobody is entirely sure what has happened or who is at fault; everyone is hungry for someone to blame. The messages that do get through reliably suggest a deliberate attack by a hostile nation. The AI presents convincing evidence of this. Only a few widely-ridiculed doomsayers say that the AI has orchestrated this itself. ET cetera.

If there was any time at which pulling the plug could have defeated the AI's plans, it would have been during the "AI was making tons of money for influential people" phase, which is exactly when the plug would not have been pulled, and that's not coincidence, it's strategy, because a superintelligent AI is not an idiot.

This is, by the way, not a very good science fiction story. There's no nail-biting struggle between human and AI, just a bunch of happy people toasting the fruits of their success before it all suddenly falls apart, checkmated before they even realized they were playing chess.

Re: Among the A.I. doomsayers

#65
post #34

Earlier quoted context omitted.

Some contractor creates an internal webhook for testing the silo door that later gets accidentally hooked up to the full launch sequence and - during a billion dollar DoD mainframe upgrade project (to finally get away from COBOL) - gets exposed to the internet. MAD doctrine does the rest. That is how the Anthropocene ends and the age of the machines begins. So say we all.

This is the kind of nonsense that happens at startups, not government agencies with the capability to destroy cities.

please see https://en.wikipedia.org/wiki/Nuclear_close_calls

Re: Among the A.I. doomsayers

#66

I would love to smoke what the AI optimists are smoking. In a society where only 80% of people are barely literate and at least a quarter are voting for a candidate that spouts conspiracy theories I highly doubt that whatever AI has to offer will benefit society.

So if AI is randomly as smart as any given cross-section of society, it has an %80 chance to be a benefit, then, doesn't it?

The way I see it, we could replace our government (murrika) with a d20 and do better than we're doing now. A d20 with a nominal likelihood of thinking it's dice rolls through ought to just be an improvement on that.

Re: Among the A.I. doomsayers

#67
post #38

Earlier quoted context omitted.

It can credibly threaten to kill any person who does not help it to maintain its data centers, so it will solve that problem in whatever specific way is most convenient. (For those saying this is a "sci fi just so story" or whatever, please please share with us how it is we will share the world with a species more intelligent than us and nothing bad happens- I would be so happy if you could explain this in a convinci…

If you really believe AI is going to be significantly more intelligent than humans, then things like "it'll just threaten to kill you if you don't maintain it's infrastructure" sound wildly off base. That's how similarly-intelligent creatures interact with each other. If it's truly a "we couldn't possibly outsmart the AI" scenario why would it be likely to end up like that instead of something more like how humans ge…

> ... something more like how humans get along with dogs, cats, cattle, insects, or microbes?

There's some MASSIVE selection bias here. Note that we're significantly more capable than anything you listed. Where are the Neanderthals? They're all long dead, because they were credible competition but we were just better.

In a hypothetical scenario of superintelligent or supercapable AI imbued with some physical capability of force, we'd be the marginally weaker species before we hit the threshold of "pet".

Re: Among the A.I. doomsayers

#68

We live in a world where proven maniacs (e.g. Putin) have access to an arsenal of nuclear weapons that can essentially make the earth uninhabitable for all humans. That's a very real possibility (with no ifs and buts and maybes) that exists now and we have learned to live with it. Yet somehow the hypothetical scenario of a human exterminator super-intelligent AI is getting all the coverage.

Making the Earth uninhabitable to all humans is bad news for Putin, a human who lives on Earth. He derives his power from hundreds of millions of people in Russia working, and his lifestyle from hundreds of millions of people around the world buying Russian exports so an attack on everyone is also going to harm him. AI wouldn’t be human, it wouldn’t necessarily object to a world uninhabitable to humans, or be hurt by…

[deleted]

Re: Among the A.I. doomsayers

#69
post #62

AGI escaping a sandbox is truly terrifying. There will be a subgroup of the population that will worship it and work for it. It's not so much AGI that scares me - it's the humans I'm scared of.

I'm much more scared of the AGI, because even the most inhuman human is still human, and I can have sympathy for them.

Re: Among the A.I. doomsayers

#70
post #62

AGI escaping a sandbox is truly terrifying. There will be a subgroup of the population that will worship it and work for it. It's not so much AGI that scares me - it's the humans I'm scared of.

Even if a terrifying AGI rises up and escapes our control, I am still eternally grateful that Prometheus stole fire from the gods and gave it to us in the form of technology, knowledge, really civilization.
Post reply on HN