Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Among the A.I. doomsayers

newyorker.com

Post reply on HN
Print view View original

41–50 of 374 posts

Re: Among the A.I. doomsayers

#41

Post by hackerlight »

bglazer wrote: post #30

In any doom scenario, like let's say >90% of humans dead, who runs the power plants that supply the AI data centers? Who runs the fabs producing more chips? Who maintains the plumbing? Presumably all these systems fall over and die within a few weeks of the AI deciding to wipe us out. Then what? The AI then dies too. It seems absurd to me that any planner capable of effortlessly destroying humanity would not see that…

ASI won't want to take over before it is capable of self-sustaining. If it did try something so stupid, it wouldn't be superintelligent.

Check out Paul Christiano's AI takeover scenario. An arms race will cause us to want to give AI control over key infrastructure and military capabilities. That's an arms race between countries and between companies. He goes into more detail.

Re: Among the A.I. doomsayers

#42

Post by drcode »

mupuff1234 wrote: post #37

It seems much more likely to me that the harm from AI will be higher unemployment rate than it will be some evil AI trying to destroy humanity.

At least some humans will survive a high unemployment rate, so let's label that problem "problem #2" and after we solve "problem #1" (human extinction) I vote we tackle that one next on the list

Re: Among the A.I. doomsayers

#43

Post by thom »

bglazer wrote: post #30

In any doom scenario, like let's say >90% of humans dead, who runs the power plants that supply the AI data centers? Who runs the fabs producing more chips? Who maintains the plumbing? Presumably all these systems fall over and die within a few weeks of the AI deciding to wipe us out. Then what? The AI then dies too. It seems absurd to me that any planner capable of effortlessly destroying humanity would not see that…

Do you think that your final sentence buys humanity a thousand years, a hundred years, or what?

Re: Among the A.I. doomsayers

#44

Post by Dalewyn »

drcode wrote: post #38
bglazer wrote: post #30

In any doom scenario, like let's say >90% of humans dead, who runs the power plants that supply the AI data centers? Who runs the fabs producing more chips? Who maintains the plumbing? Presumably all these systems fall over and die within a few weeks of the AI deciding to wipe us out. Then what? The AI then dies too. It seems absurd to me that any planner capable of effortlessly destroying humanity would not see that…

It can credibly threaten to kill any person who does not help it to maintain its data centers, so it will solve that problem in whatever specific way is most convenient. (For those saying this is a "sci fi just so story" or whatever, please please share with us how it is we will share the world with a species more intelligent than us and nothing bad happens- I would be so happy if you could explain this in a convinci…

>please please share with us how it is we will share the world with a species more intelligent than us and nothing bad happens

Don't be fucking assholes to them?

I'm serious. Of course someone (or something) will treat you like dirt if you are an asshole to them. Don't be assholes, be honorable and friendly.

And I can see your next question/argument, so I am going to preempt it: Raise your AI to be honorable and friendly and we won't have any problems.

None of this is different from how all living beings raise their offspring and build societies.

Re: Among the A.I. doomsayers

#45

Post by mitthrowaway2 »

m3kw9 wrote: post #40

Beware if someone assigns a p(doom). There is zero chance they know and 100% it came out of their backside. The only plausible p doom is a range of >0 and They really need to listen to themselves when they say there is a 50% chance we all die from AI

That's not how probability works. Probability already reflects your uncertainty. There's no "knowing the correct probability" except in very specific and rare situations.

Re: Among the A.I. doomsayers

#46

Post by esafak »

Dalewyn wrote: post #44
drcode wrote: post #38

Earlier quoted context omitted.

It can credibly threaten to kill any person who does not help it to maintain its data centers, so it will solve that problem in whatever specific way is most convenient. (For those saying this is a "sci fi just so story" or whatever, please please share with us how it is we will share the world with a species more intelligent than us and nothing bad happens- I would be so happy if you could explain this in a convinci…

>please please share with us how it is we will share the world with a species more intelligent than us and nothing bad happens Don't be fucking assholes to them? I'm serious. Of course someone (or something) will treat you like dirt if you are an asshole to them. Don't be assholes, be honorable and friendly. And I can see your next question/argument, so I am going to preempt it: Raise your AI to be honorable and frie…

That only works when there is a balance of power. Humans don't treat other animals very well -- they eat them -- even though the animals don't do anything to provoke us.

Maybe robots will become so advanced they won't have to worry about defending themselves, go explore space, and leave us alone. Who knows.

Re: Among the A.I. doomsayers

#47

Post by [deleted] »

Dalewyn wrote: post #44
drcode wrote: post #38

Earlier quoted context omitted.

It can credibly threaten to kill any person who does not help it to maintain its data centers, so it will solve that problem in whatever specific way is most convenient. (For those saying this is a "sci fi just so story" or whatever, please please share with us how it is we will share the world with a species more intelligent than us and nothing bad happens- I would be so happy if you could explain this in a convinci…

>please please share with us how it is we will share the world with a species more intelligent than us and nothing bad happens Don't be fucking assholes to them? I'm serious. Of course someone (or something) will treat you like dirt if you are an asshole to them. Don't be assholes, be honorable and friendly. And I can see your next question/argument, so I am going to preempt it: Raise your AI to be honorable and frie…

[deleted]

Re: Among the A.I. doomsayers

#48

Post by hackerlight »

matteoraso wrote: post #31

I've always wondered why people think that an AI that's super-intelligent will also be evil. It could just as likely end up being very kind (more than likely, actually, because the programmers would have safeguard to ensure that it's nice).

It's misaligned, not evil. AI doom could involve benevolent intentions (control or kill for humanity's own good), it could also involve ambivalent intentions (make more paperclips).

Re: Among the A.I. doomsayers

#49

Post by HankB99 »

If I let my imagination run wild, I can imagine that at some point AI becomes in some way sentient. By that I mean it gains reasoning, some sort of understanding, and motivation.

I wonder what the possibility is that this AI will decide that a pitched battle is going to waste resources and risk humans pulling the plug. What if it understood that and instead operated so subtly that it was not obvious it was controlling The World.

Hopefully it would not conclude that eliminating large swaths of the human population would be to its benefit.

Re: Among the A.I. doomsayers

#50

Post by anonylizard »

drcode wrote: post #38
bglazer wrote: post #30

In any doom scenario, like let's say >90% of humans dead, who runs the power plants that supply the AI data centers? Who runs the fabs producing more chips? Who maintains the plumbing? Presumably all these systems fall over and die within a few weeks of the AI deciding to wipe us out. Then what? The AI then dies too. It seems absurd to me that any planner capable of effortlessly destroying humanity would not see that…

It can credibly threaten to kill any person who does not help it to maintain its data centers, so it will solve that problem in whatever specific way is most convenient. (For those saying this is a "sci fi just so story" or whatever, please please share with us how it is we will share the world with a species more intelligent than us and nothing bad happens- I would be so happy if you could explain this in a convinci…

This. You can subjugate a population 100 times your size with only two things: Military force and accurate information.

I doubt AI will kill all humans. Its orthogonal to AI's own goals (To survive/reproduce/expand/evolve). It'll simply kill those who oppose it, and enslave/dominate the rest.

Post reply on HN