Apocalypse soon: the scientists preparing for the end times
newstatesman.com
Apocalypse soon: the scientists preparing for the end times
1–10 of 62 posts
Re: Apocalypse soon: the scientists preparing for the end times
#2Far too often, people anthropomorphize AI or draw from fiction. Bostrom does his best to dispel these notions.
The algorithm just does what it does; and unless it is a very special kind of algorithm, it does not care that we clasp our heads and gasp in dumbstruck horror at the absurd inappropriateness of its actions.
— Nick Bostrom, Superintelligence: Paths, Dangers, Strategies
This is the key point Bostrom is trying to make. If general AI research is successful, we'll need to build agents with some very carefully chosen goals. Even something as silly as "make paperclips" can result in a universe tiled with paperclips. Earth and its biomass could be turned into said paperclips, ending humanity.
Re: Apocalypse soon: the scientists preparing for the end times
#3It'd be interesting to learn the depth of study related to spectrum of scenarios in which human race will be facing apocalypse.
Few things that pop up in my head:
- disease
- nuclear (or comparable) war
- depletion of natural resources
- natural disaster
- extra terrestrial influence: asteroid or alien invasion
- artificial intelligence
- anarchy / disruption of social security
hmm...anything else?
Re: Apocalypse soon: the scientists preparing for the end times
#4It'd be interesting to learn the depth of study related to spectrum of scenarios in which human race will be facing apocalypse. Few things that pop up in my head: - disease - nuclear (or comparable) war - depletion of natural resources - natural disaster - extra terrestrial influence: asteroid or alien invasion - artificial intelligence - anarchy / disruption of social security hmm...anything else?
Gradual loss of skills and knowledge leading to a collapse of technological society.
Re: Apocalypse soon: the scientists preparing for the end times
#5It'd be interesting to learn the depth of study related to spectrum of scenarios in which human race will be facing apocalypse. Few things that pop up in my head: - disease - nuclear (or comparable) war - depletion of natural resources - natural disaster - extra terrestrial influence: asteroid or alien invasion - artificial intelligence - anarchy / disruption of social security hmm...anything else?
Anarchy won't cause an apocalypse, just a change in the structure of the ruling class as a new ruling structure is created.
also on the extra terrestrial influence side, there's direct hits by gamma rays that could wipe out Earth in an instant.
Re: Apocalypse soon: the scientists preparing for the end times
#6> “Being born into a moment when the fate of the universe is at stake is a lot of fun.”
Hubris, like heroin, offers an approximation of ecstasy. Therein lies its danger.
Re: Apocalypse soon: the scientists preparing for the end times
#7It'd be interesting to learn the depth of study related to spectrum of scenarios in which human race will be facing apocalypse. Few things that pop up in my head: - disease - nuclear (or comparable) war - depletion of natural resources - natural disaster - extra terrestrial influence: asteroid or alien invasion - artificial intelligence - anarchy / disruption of social security hmm...anything else?
Solar storm destroying a large part of the grid, leading to some heavy disruption including nuclear meltdown.
The inversion of magnetic pole could have some impact on how we are shielded from space radiation.
Re: Apocalypse soon: the scientists preparing for the end times
#8Far too often, people anthropomorphize AI or draw from fiction. Bostrom does his best to dispel these notions. The algorithm just does what it does; and unless it is a very special kind of algorithm, it does not care that we clasp our heads and gasp in dumbstruck horror at the absurd inappropriateness of its actions. — Nick Bostrom, Superintelligence: Paths, Dangers, Strategies This is the key point Bostrom is trying…
I haven't read Bostrom's Superintelligence so I may be wrong here, but this whole Evil Paper-clip AI seems like a really silly thing. First of all making paper-clips does not require super intelligence, just a simple automaton. If we however do employ AI to make paper-clip it should know how many we need on average, and what are they used for. If the paper clips are for our (human) use, the AI will be really dumb if it does not take under consideration that destroying humans to make paper clips does not make sense.
Re: Apocalypse soon: the scientists preparing for the end times
#9Far too often, people anthropomorphize AI or draw from fiction. Bostrom does his best to dispel these notions. The algorithm just does what it does; and unless it is a very special kind of algorithm, it does not care that we clasp our heads and gasp in dumbstruck horror at the absurd inappropriateness of its actions. — Nick Bostrom, Superintelligence: Paths, Dangers, Strategies This is the key point Bostrom is trying…
I haven't read Bostrom's Superintelligence so I may be wrong here, but this whole Evil Paper-clip AI seems like a really silly thing. First of all making paper-clips does not require super intelligence, just a simple automaton. If we however do employ AI to make paper-clip it should know how many we need on average, and what are they used for. If the paper clips are for our (human) use, the AI will be really dumb if…
Obviously we don't actually need a superintelligence to make paper clips, it's just an example to illustrate that an AI won't do what you want it to do if you program its motivations incorrectly. If you haven't programmed an AI to place sufficient importance on human life, and then tell it to maximize paperclip production, it could take that as its primary goal and see humans as made from raw material it could use to make more paperclips. It may know that you want the paper clips for human use, but does it have any reason to care if it hasn't been made with that as part of its goal system?
Re: Apocalypse soon: the scientists preparing for the end times
#10Earlier quoted context omitted.
I haven't read Bostrom's Superintelligence so I may be wrong here, but this whole Evil Paper-clip AI seems like a really silly thing. First of all making paper-clips does not require super intelligence, just a simple automaton. If we however do employ AI to make paper-clip it should know how many we need on average, and what are they used for. If the paper clips are for our (human) use, the AI will be really dumb if…
Obviously we don't actually need a superintelligence to make paper clips, it's just an example to illustrate that an AI won't do what you want it to do if you program its motivations incorrectly. If you haven't programmed an AI to place sufficient importance on human life, and then tell it to maximize paperclip production, it could take that as its primary goal and see humans as made from raw material it could use to…
I don't think AI will be "programmed" in a traditional sense of this word. AI will need to go through a learning process, where it will have a chance to learn all about humans, and the environment. Simple Paper Clip AI (SPCAI) will not be smart enough, nor powerful enough to prevent us from stopping it when we see that it's starting to make paper clips out of humans. True AI (tm) on the other hand will be smart enough to know that making paper clips out of humans is simply stupid.
True AI may decide that the humans are unnecessary and remove us for other reasons. I'm sorry but if the paper clip AI doomsday scenario is the best this institute could come up with to explain why AI could be dangerous then I'm not impressed.