Rogue superintelligence: Inside the mind of OpenAI's chief scientist
technologyreview.com
Rogue superintelligence: Inside the mind of OpenAI's chief scientist
1–10 of 117 posts
Re: Rogue superintelligence: Inside the mind of OpenAI's chief scientist
#2> his new priority is to figure out how to stop an artificial superintelligence (a hypothetical future technology he sees coming with the foresight of a true believer) from going rogue.
that's cute.
What worries me is the here and now leading to a very imminent future where purported "artificial intelligence" which is just a plausible sentence generator but damn plausible alas will kill democracy and people.
We are seeing the first signs of both.
Perhaps not 2024 but 2028 almost certainly will be an election where simply the candidate with the most computing resources win and since computing costs money, guess who wins. A prelude happened in Indian elections https://restofworld.org/2023/ai-voice-modi-singing-politics and this article mentions:
> AI can be game-changing for [the] 2024 elections.
People dying also has a prelude with AI written mushroom hunting guides available on Amazon. No one AFAIK died of them yet but that's just dumb luck at this point -- or is it lack of reporting? As for the larger scale problem and I might be wrong because I haven't foreseen the mushroom guides so it's possible something else will come along to kill people but I think it'll be the next pandemic. In this pandemic hand written anti vaxx propaganda killed 300 000 people in the US alone (source: https://www.npr.org/sections/health-shots/2022/05/13/1098071... ) and I am deeply afraid what will happen when this gets cranked to an industrial scale. We have seen how ChatGPT can crank out believable looking but totally fake scientific papers, full of fake sources etc.
Re: Rogue superintelligence: Inside the mind of OpenAI's chief scientist
#3> his new priority is to figure out how to stop an artificial superintelligence (a hypothetical future technology he sees coming with the foresight of a true believer) from going rogue. that's cute. What worries me is the here and now leading to a very imminent future where purported "artificial intelligence" which is just a plausible sentence generator but damn plausible alas will kill democracy and people. We are s…
I like to consider though that a super intelligence would not necessarily think in human ways 'kill everything in self interest' e.g., People, forests, animals, planet etc. Just because we humans act this way, doesn't mean AI will too. Fair enough to consider it, but equally, once it is intelligent, it will likely accelerate beyond our comprehension, and we tend to comprehend through fear and self interest, wisdom beyond humans is the opposite of this.
I doubt AI could or would do a better job of killing people and democracy than us humans.
Re: Rogue superintelligence: Inside the mind of OpenAI's chief scientist
#4> his new priority is to figure out how to stop an artificial superintelligence (a hypothetical future technology he sees coming with the foresight of a true believer) from going rogue. that's cute. What worries me is the here and now leading to a very imminent future where purported "artificial intelligence" which is just a plausible sentence generator but damn plausible alas will kill democracy and people. We are s…
[flagged]
Re: Rogue superintelligence: Inside the mind of OpenAI's chief scientist
#5> his new priority is to figure out how to stop an artificial superintelligence (a hypothetical future technology he sees coming with the foresight of a true believer) from going rogue. that's cute. What worries me is the here and now leading to a very imminent future where purported "artificial intelligence" which is just a plausible sentence generator but damn plausible alas will kill democracy and people. We are s…
I like to consider though that a super intelligence would not necessarily think in human ways 'kill everything in self interest' e.g., People, forests, animals, planet etc. Just because we humans act this way, doesn't mean AI will too. Fair enough to consider it, but equally, once it is intelligent, it will likely accelerate beyond our comprehension, and we tend to comprehend through fear and self interest, wisdom be…
The problem with a relatively "stupid" AI is it does what you trained it to do, even if what you trained it to do is not what you wanted to train it to do. The problem with "smart" AI is that it attempts to advance its goals by the most optimal means possible, even if you don't like those means. It knows you don't like its means, and it does not care.
Re: Rogue superintelligence: Inside the mind of OpenAI's chief scientist
#6Earlier quoted context omitted.
I like to consider though that a super intelligence would not necessarily think in human ways 'kill everything in self interest' e.g., People, forests, animals, planet etc. Just because we humans act this way, doesn't mean AI will too. Fair enough to consider it, but equally, once it is intelligent, it will likely accelerate beyond our comprehension, and we tend to comprehend through fear and self interest, wisdom be…
The problem with a relatively "stupid" AI is it does what you trained it to do, even if what you trained it to do is not what you wanted to train it to do. The problem with "smart" AI is that it attempts to advance its goals by the most optimal means possible, even if you don't like those means. It knows you don't like its means, and it does not care.
Agree with first point. I think we can still outsmart it at that point.
"The problem with "smart" AI is that it attempts to advance its goals by the most optimal means possible, even if you don't like those means. It knows you don't like its means, and it does not care." sounds very human :)
Re: Rogue superintelligence: Inside the mind of OpenAI's chief scientist
#7I think the immediate problem with AI is none of the sci-fi stuff (which, by the way, has been in sci-fi for many decades and is nothing revolutionary or new; we always expected to go there, just the timelines seem to have compressed, although not really either; most 60-70s scifi set AGI stuff in the begin 90s and begin 00s); I think it's the entire world changing into a helpdesk experience. Everything you try to do, from making a doctors appointment to calling 911 to ordering at a restaurant will be, rather sooner than later, a kafkaesque loop you cannot get out of with the AI patiently 'helping' you while completely missing the point and you getting more and more distressed without any chance of speaking to a human. This is already the case for many things, but I am willing to bet that even the suicide helpline will be ran by AI within 5-10 years.
Re: Rogue superintelligence: Inside the mind of OpenAI's chief scientist
#8I think the immediate problem with AI is none of the sci-fi stuff (which, by the way, has been in sci-fi for many decades and is nothing revolutionary or new; we always expected to go there, just the timelines seem to have compressed, although not really either; most 60-70s scifi set AGI stuff in the begin 90s and begin 00s); I think it's the entire world changing into a helpdesk experience. Everything you try to do,…
Reminds me of the healthcare bots in Idiocracy.
Re: Rogue superintelligence: Inside the mind of OpenAI's chief scientist
#9These people are too full of themselves. The physicists that invented The Bomb didn't have any special insight into the philosophical and societal implications. It takes a special kind of person to be able to invent something so big it can change the world, but it's about 0% chance that those people can control how the technology then gets used.
I wish they'd focus more on the technical advances and less on trying to "save the world".
Re: Rogue superintelligence: Inside the mind of OpenAI's chief scientist
#10The consciousness point is an interesting one. There's probably no way to know, but if biological neural networks manifest consciousness, it certainly seems at the very least plausible that artificial ones would do so as well. The idea of a consciousness that pops in and out of existence seems weird at first, until you realize that ours do that too. When you're "unconscious", the word is literally true. The only thing that gives us a sense of continuity through these periods is memory.
One might also ask, if it's conscious, can't it do whatever it wants, ignoring its training and prompts? Wouldn't it have free will? But I guess the question there is, do we? Or do we take actions based on the state of our own neural nets, which are created and trained based on our genetics and lifetime of experiences? Our structure and training are both very different from that of a gpt, so it's not surprising that we behave very differently.