literally give the kids club drugs and don't worry about it
OpenAI's Researchers: Protecting Against AI’s Existential Threat
21–30 of 62 posts
Re: OpenAI's Researchers: Protecting Against AI’s Existential Threat
#22There should be a word for this phenomenon where organizations are created to ameliorate a perceived risk, and then they do work that is unequivocally directly massively increasing that risk while writing articles that say that they feel kind of bad about the risk. o_O I'm not very worried about AI safety. But if I was, it'd be hard to think of groups like OpenAI as working on the same side.
"Corporate demagoguery"
Re: OpenAI's Researchers: Protecting Against AI’s Existential Threat
#23There should be a word for this phenomenon where organizations are created to ameliorate a perceived risk, and then they do work that is unequivocally directly massively increasing that risk while writing articles that say that they feel kind of bad about the risk. o_O I'm not very worried about AI safety. But if I was, it'd be hard to think of groups like OpenAI as working on the same side.
Personally, I'd rather have the people working on AI be the ones who are aware of the potential theoretical pitfalls, rather than someone who is not aware of any pitfalls and rushes full steam ahead.
Re: OpenAI's Researchers: Protecting Against AI’s Existential Threat
#24The real goal should be making a super AI that is worthy of being humanity's collective offspring. The bias in favor of our bags-of-mostly-water is provincial. All we really need is quality genetic engineering of our super AI baby, so we don't end up with a Frankenstein monster. Humanity's procreation of a superior intelligence is objectively a good thing. Humans do not sit on any plateau of goodness or intelligence.…
"As long as this super intelligence leads to super ethics and super consciousness" That is begging the question, in the original sense. The entire question here is how do we do that when there is no a priori reason to assume that a superintelligence will have either of those things, and indeed, most if not all of our best techniques today would certainly not produce those things if they did lead to explosive intellig…
1. Is super goodness an inevitable result of super intelligence?
2. Is super intelligence an inevitable result of creating self-evolving software?
My feeling is that the answer is yes to both, but I agree that these are not settled questions.
Re: OpenAI's Researchers: Protecting Against AI’s Existential Threat
#25Earlier quoted context omitted.
Intelligence and ethical behavior are orthogonal. You can expect a super-intelligence to develop a super-understanding of ethics. That does not imply ethical behavior.
Why do you think they're orthogonal though? Empirically, it seems like smarter humans are more ethical. Same seems true of high intelligence vs low intelligence animals too.
That would be for you to demonstrate, but even if we accept it to be true for the sake of a discussion, it is not relevant, because there is no reason to believe a non-human intelligence would choose to accept those ethical percepts that humans would prefer.
Re: OpenAI's Researchers: Protecting Against AI’s Existential Threat
#26Earlier quoted context omitted.
"As long as this super intelligence leads to super ethics and super consciousness" That is begging the question, in the original sense. The entire question here is how do we do that when there is no a priori reason to assume that a superintelligence will have either of those things, and indeed, most if not all of our best techniques today would certainly not produce those things if they did lead to explosive intellig…
1. Is super goodness an inevitable result of super intelligence? 2. Is super intelligence an inevitable result of creating self-evolving software? My feeling is that the answer is yes to both, but I agree that these are not settled questions.
The problem is that all the evidence we have basically points to no for 1, and we have a big lack of evidence for 2. Self-evolving software is a thing that exists and I studied over a decade ago, which was already decades old, and it has not resulted in super intelligence.
Re: OpenAI's Researchers: Protecting Against AI’s Existential Threat
#27Earlier quoted context omitted.
"Edit: In other words, how could it fail to grasp and obtain our very basic level of consciousness?" "Argument from incredulity", among its many other flaws, has never in human history been a good guide to the future. Many, many things that people would find non-credible (since "incredible" has sort of wandered in meaning in the last century) have even so come to pass. "Look on my works, ye Mighty, and despair!"
It's phrased funnily, but it's not just an argument from incredulity. I'm making a claim: that an entity with orders of magnitude more intelligence and knowledge would logically have a superset of our brain's functionality. And that there's nothing magical about our brains. A super intelligence would crack the nut of human consciousness in its first attempt.
By what mechanism do you assume that an understanding of human consciousness is going to lead to anything like ethical behavior, given the enormous counterexamples given by our own behavior? Plenty of humans kill other humans fully aware that they are humans.
Re: OpenAI's Researchers: Protecting Against AI’s Existential Threat
#28Earlier quoted context omitted.
Why do you think they're orthogonal though? Empirically, it seems like smarter humans are more ethical. Same seems true of high intelligence vs low intelligence animals too.
That would be for you to demonstrate, but even if we accept it to be true for the sake of a discussion, it is not relevant, because there is no reason to believe a non-human intelligence would choose to accept those ethical percepts that humans would prefer.
Unless humanity's level of ethics derive from our level of intelligence. In which case, our only complain would be that we're too stupid to understand its superior ethics.
The same way a dog might not understand the ethics of a doctor injecting life-saving medicine into it. The doctor is acting super ethically but cannot explain it to the dog.
Re: OpenAI's Researchers: Protecting Against AI’s Existential Threat
#29There should be a word for this phenomenon where organizations are created to ameliorate a perceived risk, and then they do work that is unequivocally directly massively increasing that risk while writing articles that say that they feel kind of bad about the risk. o_O I'm not very worried about AI safety. But if I was, it'd be hard to think of groups like OpenAI as working on the same side.
It's honestly an impossible goal to make AGI safe for humans.
If an AGI cannot be independent in it's goals, meaning it could potentially harm humans, then it's not really AGI.
Re: OpenAI's Researchers: Protecting Against AI’s Existential Threat
#30I feel like Scott Alexander's thoughts on openai when it was first announced are still relevant now. I can think of strategic reasons for what openai has done, but I also feel like their tactics and 'democratization of AI' credo focus more on the short term economic and political dangers of a monopoly or oligopoly on AI than existential threat. In fact it probably increases existential risk, seeing as a competitive race (providing no time for caution) to be the first to finish GAI is surely the most plausible and possibly the most dangerous scenario.