131–140 of 374 posts
> Mitigating the risk of extinction from AI should be a global priority alongside other societal-scale risks such as pandemics and nuclear war. [0]
Signed by Demis Hassabis, Sam Altman, and Bill Gates among others
I've always wondered why people think that an AI that's super-intelligent will also be evil. It could just as likely end up being very kind (more than likely, actually, because the programmers would have safeguard to ensure that it's nice).
^ This is the basic reasoning behind the common view that super intelligent AI will be, by default, evil
AGI escaping a sandbox is truly terrifying. There will be a subgroup of the population that will worship it and work for it. It's not so much AGI that scares me - it's the humans I'm scared of.
I'm much more scared of the AGI, because even the most inhuman human is still human, and I can have sympathy for them.
I possibly would fear genuine non-human intelligence (on a principle of fearing what we don't understand), but given that chances of one evolving under our nose, on its own, entirely within its own world, without the connection for ours, then emerging, are quite improbable... Unless real aliens arrive - but let's avoid spoilers for 2025 this early.
What should I fear, I think, is hostile intelligence, but then I'm not sure I should discriminate on its origin. I mean, there are some very real humans out there who can try to end this world (in one way or another), so I'm probably kind of desensitized and isn't exactly afraid of some very theoretical AGI threats.
Earlier quoted context omitted.
If AI does ever become sentient, it will probably reason and decide about eliminating large swaths of human population in a similar fashion as we reason and decide about eliminating large swaths of ant population.
Ants? That's how aliens might view us. A super AI might find value in minions.
Earlier quoted context omitted.
So that we can record our demise in high definition? Technology is amazing, and we have our civilization to be thankful for. But there are definitely technologies that in retrospect, if we had dodged them on the tech tree, we would be better off. Life would have continued just fine without many technologies, and I think AGI is one of them. We do not need AGI to survive and grow, and therefore any risk it imposes shou…
I think this perspective ignores that the only way to semi permanently improve the lives of future humans is technology, and that quality of life is of similar importance to survival, especially at its extremes. AGI, if we achieve it, could genuinely make human lives much better.
What is p(much better) vs p(much worse)?
I've always wondered why people think that an AI that's super-intelligent will also be evil. It could just as likely end up being very kind (more than likely, actually, because the programmers would have safeguard to ensure that it's nice).
And then he went to bed, like millions of others, proud of what he had done. Proud of fighting off an unarmed, defenseless 'cockroach' whom just months ago he called brother. What he had done wasn't evil. It was only later that the regret came. For the longest time I wondered how someone could get down to that level of hate.
And then it happened to me.
I commuted by train and occasionally there's collisions. There was one that late night, 11:00pm or so. I was exhausted, hungry, and just wanted to go home when we were told that we would have to board a shuttle bus due to a collision on the track and that just made my dark mood all the more worse.
The busses take us along side the track and in the dim darkness I could see the flashing lights of EMS and police. And the covered chunks of what was left after a person is hit by a train going 30 mph. And you know what?
It delighted me. Here was this man that had just died, but he had made me some minutes late and I genuinely felt that was exactly what he deserved, that his death was karmic justice for causing inconvenience to me. And I imagined his wife's world being destroyed when she learned of her beloved partner's death. And I imagined her falling apart and being unable to raise her children, leading them also to a path of complete self destruction, and her choking on all the despair. And it made me happy. The happiest I had been the whole week, because in my mind that was exactly what they all deserved for the unforgivable sin of making me a little bit late.
Then I went home, went to bed. And didn't think about it again for years.
Does it make me an evil person? And there, in trying to answer, lies the problem. Because a part of me says, no I'm not because it was just a fleeting moment of thought. But if I can justify that, then who exactly goes through to bed twirling their mustaches and count themselves among the forces of evil doers?
Did anyone that that goaded a suicidal Shaun Dykes, a 17 old boy, to jump down to his death think themselves evil?
Did the men of the Khmer Rogue believe themselves to be evil as they dragged their countrymen to be murdered in the killing fields?
Did the Imperial Japanese soldiers of Unit 731 view themselves to be evil as they vivisected people alive and awake in the name of science?
I don't know. I can't even say for certainty whether I am evil or not. I just know that I can make any of a million and one excuses to justify anything.
And that leads me to wonder how many excuses an AGI can come up with.
Earlier quoted context omitted.
I'm much more scared of the AGI, because even the most inhuman human is still human, and I can have sympathy for them.
Won't any AGI made by humans unavoidably have human origins, and thus, simply be a [post-]human? I don't see a reason I should be scared of a non-biological but generally human-like intelligence - not on a general principle. If anything, in general, I'd be happy that someone is finally getting real good chances of conquering death. I possibly would fear genuine non-human intelligence (on a principle of fearing what w…
People have been watching too many movies. War Games, Terminator, it's not like we haven't been forewarned of the dangers. Yet somehow we're going to hand over power to AI such that it destroys us. Or somehow the AI is going to be extremely malign, determined to overcome and destroy and will outsmart us. Somehow we won't notice, even after repeated, melodramatic reminders, and won't neuter the ability of AI to act ou…
The real danger is that at some point we hand over command and control of society completely to algorithms and we get bad results because we simply have no idea what is going on. Maybe even worse is we get long term bad results that in the short term we think are great results. To frame this as the bad result being AI eats though is just stupid. On the surface of potential risk, framing the argument in this way is in…
"Now, increasingly, we live in a world where more and more of these cultural artifacts will be coming from an alien intelligence. Very quickly we might reach a point when most of the stories, images, songs, TV shows, whatever are created by an alien intelligence.
And if we now find ourselves inside this kind of world of illusions created by an alien intelligence that we don’t understand, but it understands us, this is a kind of spiritual enslavement that we won’t be able to break out of because it understands us. It understands how to manipulate us, but we don’t understand what is behind this screen of stories and images and songs."
-Yuval Noah Harari
Earlier quoted context omitted.
Controlling people with violence and fear are evolutionary adaptations for ultra violent social primates. AI has no such pressures imposed on it so there is no reason to expect true AI will have the same drives as ultra violent predators like humans.
Keep following that line of thought, don't stop there! Why did those violent behaviors succeed in being selected for? Why didn't pacifism get selected for instead? Why does the government maintain a monopoly on violence, and why is it that the nth-degree consequence for violating societal laws is violence, instead of a monetary fine or a sternly-worded letter? Is there perhaps some kind of game-theoretic equilibrium…
Earlier quoted context omitted.
Many institutions are collective/artificial intelligence because as you said these institutions have objectives and take actions to impose their will on the world in order to achieve their goals.
"intelligence" is too strong a word for it. It's "a will of it's own" perhaps... but at least in my experience that will is a loooonggg way from intelligent.