No post body was provided.
Ask HN: In simple terms, how could AI destroy mankind?
1–10 of 16 posts
Re: Ask HN: In simple terms, how could AI destroy mankind?
#2AI would more likely destroy mankind by screwing up rather than by becoming conscious. Perhaps something to do with power and energy supply. Like a stuxnet kind of thing.
Re: Ask HN: In simple terms, how could AI destroy mankind?
#3Simply, by it doing what you tell it but not the way you expect.
It's a little contrived but you tell it 'solve world hunger', so it 'does a Thanos' and wipes out half the human population by releasing a pathogen or something, so it's fulfilled it's primary function but (hopefully) not in the way you expected.
Re: Ask HN: In simple terms, how could AI destroy mankind?
#4I think the real threat is going to be more of a side-effect of AI. The fact that the current models may produce results that we can't foresee; therefore, a sufficiently advanced AI is powerful AND unpredictable, hence uncontrollable.
Re: Ask HN: In simple terms, how could AI destroy mankind?
#5Simply, by it doing what you tell it but not the way you expect. It's a little contrived but you tell it 'solve world hunger', so it 'does a Thanos' and wipes out half the human population by releasing a pathogen or something, so it's fulfilled it's primary function but (hopefully) not in the way you expected.
If you were to define a clear goal which the AI strives for, wouldn't it be possible to define other goals along with it such as "Never ever hurt humans"?
Re: Ask HN: In simple terms, how could AI destroy mankind?
#6AI would more likely destroy mankind by screwing up rather than by becoming conscious. Perhaps something to do with power and energy supply. Like a stuxnet kind of thing.
Stuxnet was a well-programmed virus rather than an AI-driven one.
Re: Ask HN: In simple terms, how could AI destroy mankind?
#7AI would not think in the same timescales that people use, so something like killing sea life with plastic drinking straws or changing the climate via herbivore flatulence - though lengthy by our standards - might make logical, reasonable sense on its part as tools of human extinction.
Re: Ask HN: In simple terms, how could AI destroy mankind?
#8Simply, by it doing what you tell it but not the way you expect. It's a little contrived but you tell it 'solve world hunger', so it 'does a Thanos' and wipes out half the human population by releasing a pathogen or something, so it's fulfilled it's primary function but (hopefully) not in the way you expected.
If you were to define a clear goal which the AI strives for, wouldn't it be possible to define other goals along with it such as "Never ever hurt humans"?
That's not a clear goal. For example, define "hurt". People define hurt in all kinds of ways, and sometimes differently when it's themselves or someone else.
Then there's the problem that humans hurt other humans. Should the AI stop that? It's going to have to hurt humans to do it. But if it doesn't, that will hurt other humans...
Re: Ask HN: In simple terms, how could AI destroy mankind?
#9We could listen to it. But an AI could have good-sounding but insane reasoning that leads to insane results if we follow its recommendations. And, if the AI were more advanced than we are, we couldn't tell. We could only trust it, or not. But if we trust it and it's wrong...
A more malevolent AI could hack its way into infrastructure. Even if we intended to leave it airgapped, it could probably find a way around it (we humans seem to be really bad at true airgapping). From there, it could destroy, not mankind, but civilization and most of the human race.
Re: Ask HN: In simple terms, how could AI destroy mankind?
#10Simply, by it doing what you tell it but not the way you expect. It's a little contrived but you tell it 'solve world hunger', so it 'does a Thanos' and wipes out half the human population by releasing a pathogen or something, so it's fulfilled it's primary function but (hopefully) not in the way you expected.
If you were to define a clear goal which the AI strives for, wouldn't it be possible to define other goals along with it such as "Never ever hurt humans"?
wouldn't it be possible to define other goals along with it such as "Never ever hurt humans"?
I'm not even close to being an AI alarmist, and I'm skeptical of a lot of Nick Bostrom's arguments. But he does do a pretty good job of articulating the problem with this scenario in his book Superintelligence. He makes a good case that it would be very difficult to articulate such values for the AI. If you're interested in this topic in the general sense, I'd suggest reading the book. I don't think it's perfect, but I will acknowledge that he makes some good points.