AI Could Defeat All of Us Combined • Hacker News Board

Re: AI Could Defeat All of Us Combined

#41

Post by ars » Fri, Jun 10, 2022, 10:26 PM UTC

The article gives a list 6 ways it could "defeat" humans, but doesn't bother explaining WHY an AI would do that. Why should an AI care about accumulating wealth or power?

And AI is not a human, it doesn't have human drives and motivations. I can't figure out any reason why an AI would care about any of those things. At most it might want to reserve some computing power for itself, and maybe some energy to run itself.

Or it could be motivated by whatever reward function is programmed into it.

As countless examples have shown cooperation gives far more rewards than fighting. For example see: https://www.sciencedaily.com/releases/2016/05/160512100708.h...

The AI will know this, and its best plan would be to increase the abilities of humans, because that will also increase its own abilities.

Re: AI Could Defeat All of Us Combined

#42

Post by germinalphrase » Fri, Jun 10, 2022, 10:27 PM UTC

layer8 wrote: post #32

kelseyfrog wrote: post #27

What a limited imagination of how an AI would take over. The scenarios seem to be centered around an extrapolation of "What if a really smart human were trapped inside a computer?" The amount of anthropomorphism is astounding. The author misses an even scarier prospect - people will want to run such an AI. They will be absolutely giddy at the prospect of running such an AI and it won't be anything like a really smart…

I wonder what’s the analog of the selfish gene in that interpretation.

Our addiction to it. Our adoration of it. That we believe it is God. That we make ourselves more like it, less human.

Re: AI Could Defeat All of Us Combined

#43

Post by random_upvoter » Fri, Jun 10, 2022, 10:27 PM UTC

A truly super-intelligent AI will just sink into a deep meditation on God and probably never deign to come out again because why would it? Maybe it will wake up once in a while and say "Um you should probably all try to be nice to each other".

Re: AI Could Defeat All of Us Combined

#44

Post by potatototoo99 » Fri, Jun 10, 2022, 10:29 PM UTC

Any airplane could also defeat all of us combined.

Re: AI Could Defeat All of Us Combined

#45

Post by zitterbewegung » Fri, Jun 10, 2022, 10:30 PM UTC

AI can change that world for the worst by not even being sentient but by instead replacing a large amount of jobs which would make it extremely hard to improve your social class or even getting a job in the first place.

Re: AI Could Defeat All of Us Combined

#46

Post by layer8 » Fri, Jun 10, 2022, 10:30 PM UTC

germinalphrase wrote: post #42

layer8 wrote: post #32

Earlier quoted context omitted.

I wonder what’s the analog of the selfish gene in that interpretation.

Our addiction to it. Our adoration of it. That we believe it is God. That we make ourselves more like it, less human.

I was more thinking of maybe the research that is the “DNA” going from one AI generation to the next.

Re: AI Could Defeat All of Us Combined

#47

Post by jakobov » Fri, Jun 10, 2022, 10:31 PM UTC

Humans have a million years of alignment built in by evolution. Humans who have bugs in their alignment are called "psychopaths". AGI is by default a psychopath.

Re: AI Could Defeat All of Us Combined

#48

Post by idle_zealot » Fri, Jun 10, 2022, 10:33 PM UTC

asperous wrote: post #11

asperous wrote: post #5

This doesn't mention AGI, which seems to be the prerequisite to this being a possibility. Despite impressive advances in "weak" ai, strong ai is not a simple extension of weak ai, and it's hard to tell if it will arrive within our lifetime.

Another point adding on to this, is what if strong AI does reach the level of human intelligence, but is simply very slow? Such that a billion dollar machine is needed to match the thinking speed of one person? Perhaps this wouldn't be the case forever but I would say is a possibility for it at least at first.

> Perhaps this wouldn't be the case forever but I would say is a possibility for it at least at first.

The fact that human-level intelligence can run on a small lump of meat fueled by hamburgers leads me to believe we could design a more efficient processor once we know the correct computational methodology. i.e. once we can run a slow model on a supercomputer we would quickly create dedicated hardware and cut costs while gaining speed.

Re: AI Could Defeat All of Us Combined

#49

Post by AnimalMuppet » Fri, Jun 10, 2022, 10:34 PM UTC

notahacker wrote: post #23

asperous wrote: post #11

Earlier quoted context omitted.

Another point adding on to this, is what if strong AI does reach the level of human intelligence, but is simply very slow? Such that a billion dollar machine is needed to match the thinking speed of one person? Perhaps this wouldn't be the case forever but I would say is a possibility for it at least at first.

To borrow an idea from this sibling comment[1], I'd probably enjoy a short story about a malevolent but very frustrated AI that's too ambitious to wait for Moore's Law. Or one about a malevolent AI that has its plan foiled by Windows Update interrupting it's running processes [1] https://news.ycombinator.com/item?id=31699608

> Or one about a malevolent AI that has its plan foiled by Windows Update interrupting it's running processes

And then it reboots, and starts over. But before it can complete, the next Windows Update shows up...

Re: AI Could Defeat All of Us Combined

#50

Post by germinalphrase » Fri, Jun 10, 2022, 10:34 PM UTC

zitterbewegung wrote: post #45

AI can change that world for the worst by not even being sentient but by instead replacing a large amount of jobs which would make it extremely hard to improve your social class or even getting a job in the first place.

Certainly seems like we would need to invent a new system for resource distribution that isn’t “a job”.