nice addition
Introducing OpenAI
261–270 of 392 posts
Re: Introducing OpenAI
#262> Musk: I think the best defense against the misuse of AI is to empower as many people as possible to have AI. If everyone has AI powers, then there’s not any one person or a small set of individuals who can have AI superpower. In a sense, we have no other defense. AI is just math and code, and I know of no way to distinguish good linear algebra from evil linear algebra. The barriers to putting that math and code tog…
Ecosystems are hellish malthusian processes rife with extinction. I'm unsure why they think an ecosystem of superhuman agents will be any different. What sort of selective pressures would exist in such a world that would ensure the survival of humanity? It's pretty naive to think a competitive ecology would select for anything more than very intelligent entities that value replication.
Physics. You're right that ecosystems are brutal. That's exactly why I'm not worried about AI as an existential threat to humanity.
A few years back Bill Joy was sounding the alarm on nanotechnology. He sounded a lot like Elon Musk does today. Nanobots could be a run-away technology that would reduce the world to "grey goo". But nothing like that will ever happen. The world is already awash in nanobots. We call them bacteria. Given the right conditions, they grow at an exponential rate. But they don't consume the entire earth in a couple of days, because "the right conditions" can't be sustained. They run out of energy. They drown in their own waste.
AI will be the same. Yes, machines are better than us at some things, and that list is growing all the time. But biology is ferociously good at converting sunlight into pockets of low entropy. AI such as it exists today is terrible at dealing with the physical world, and only through a tremendous amount of effort are we able to keep it running. If the machines turn on us, we can just stop repairing them.
Re: Introducing OpenAI
#263> Musk: I think the best defense against the misuse of AI is to empower as many people as possible to have AI. If everyone has AI powers, then there’s not any one person or a small set of individuals who can have AI superpower. In a sense, we have no other defense. AI is just math and code, and I know of no way to distinguish good linear algebra from evil linear algebra. The barriers to putting that math and code tog…
Ecosystems are hellish malthusian processes rife with extinction. I'm unsure why they think an ecosystem of superhuman agents will be any different. What sort of selective pressures would exist in such a world that would ensure the survival of humanity? It's pretty naive to think a competitive ecology would select for anything more than very intelligent entities that value replication.
There's a very real case that any superhuman agent would prioritize its own survival over the survival of any human, since (presumably) its utility function would encourage it to continually guarantee the survival of "humanity" or something larger than any single human... and it can't guarantee anything if it's dead, right?
So the moment there's a war between two superhuman agents, either of them could end up de-prioritizing human life, more so than they might if either of them existed in isolation.
And if there's actually a starvation of resources if there are a large number of superhuman agents? Am I missing something obvious here?
If I'm not missing anything... it's painfully ironic to me that we worry about the AI Box, and yet by open-sourcing the work of the best minds in AI, we voluntarily greatly increase the probability that AI will be born outside of a box - somebody is going to be an idiot and replicate the work outside of a sandbox.
Now, despite all of this, I'm optimistic about AI and its potential. But I believe the best chance we have is when the best, most well-intentioned researchers among us have as much control as possible over designing AI. Ruby on Rails can be used to make fraudulent sites, but it's fine to open-source it since fraudulent sites don't pose an existential risk to sentient biological life. That is not necessarily the case here.
Re: Introducing OpenAI
#264>unconstrained by a need to generate financial return
The incentive, not the constraint, provided by financial return is what drives innovation the most, aside from (but not mutually exclusive to) necessity.
Re: Introducing OpenAI
#265> Musk: I think the best defense against the misuse of AI is to empower as many people as possible to have AI. If everyone has AI powers, then there’s not any one person or a small set of individuals who can have AI superpower. In a sense, we have no other defense. AI is just math and code, and I know of no way to distinguish good linear algebra from evil linear algebra. The barriers to putting that math and code tog…
The risk of AI comes from the control problem. We have a vague idea on how to build AIs. The work is just to optimize the algorithms extensively so they can run in real time. But we have no idea how to control those AIs. We can push a "reward button" every time it does something we like. But we can't prevent it from killing the programmer and stealing the button for itself.
It doesn't matter at all who makes AI first. Without the control problem solved, no one will be able to use it for evil ends, even if they want to. But it's still incredibly dangerous if it gets loose in some uncontrolled setting.
So this project is the exact opposite of what you would want to do if you wanted to minimize AI risk. This is rushing towards the risk as fast as possible.
Re: Introducing OpenAI
#266Man I dunno about some of this media hype surrounding the topic of AI. I understand how powerful ML/AI algorithms are for general pattern matching (with a big enough computer, gradient descent can learn a lot of things...), but this whole skynet/doomsday fear thing seems ridiculous. I guess the risk is embedding into systems that manage missiles or something. But you don't need sophisticated algorithms for that to be…
In my opinion the biggest danger is letting AI to sort our news and search engine results, social media feeds etc. There was research on Facebook how they can affect people's moods by using different weights for posts. What happens when intelligent bots start writing news, blogs, comments, tweets? In essence, I mean the dangers of using AI for large scale propaganda through Internet services. The best tools of the mo…
- Do you know you are an ad? south park ;)
I doubt it though. People got quickly desensitized towards advertising. Propaganda will follow.
Re: Introducing OpenAI
#267Earlier quoted context omitted.
You are not correct, there are people working on these problems who are experts in the relevant technical fields. Just not enough of them. But yes, I'm also concerned about the lack of safety-focused headliners at OpenAI, given the message that they think safety is important.
Who are those researchers? I'll admit I don't follow the stuff written by the friendly AI folks very much; I only know of Bostrom/Yudkowsky, both of whom are very much philosophers. All of the hype around ML today is in deep learning (let's be honest, OpenAI would not exist if that wasn't the case), and AFAIk there is almost no overlap between people who are prolific in deep learning and prolific in FAI.
You won't find direct ML work from MIRI because:
1. AI / ML is not AGI.
2. Deep learning may be a tool used by an AGI, but is not itself capable of becoming an AGI.
3. MIRI believes it would be irresponsible to build, or make a serious effort at building, an AGI before the problem of friendliness / value alignment is solved.
So are they philosophers? Of a sort, but at least Eliezer is one who can do heavy math and coding that most engineers can't. I wouldn't have an issue calling him a polymath.
There are lots of individuals who disagree to various extents on point 3. Pretty much all of them are harmless, which is why MIRI isn't harping about irresponsible people. But the harmless ones can still do good work on weak AI. You should look up people who were on the old shock level 4 mailing list. Have a look into Ben Goertzel's work (some on weak AI, some on AGI frameworks) and the work of others around OpenCog for an instance of someone disagreeing with 3 who nevertheless has context to do so. Also be sure to look up their thoughts if they have any on deep learning.
Re: Introducing OpenAI
#268Earlier quoted context omitted.
Ecosystems are hellish malthusian processes rife with extinction. I'm unsure why they think an ecosystem of superhuman agents will be any different. What sort of selective pressures would exist in such a world that would ensure the survival of humanity? It's pretty naive to think a competitive ecology would select for anything more than very intelligent entities that value replication.
Physics. You're right that ecosystems are brutal. That's exactly why I'm not worried about AI as an existential threat to humanity. A few years back Bill Joy was sounding the alarm on nanotechnology. He sounded a lot like Elon Musk does today. Nanobots could be a run-away technology that would reduce the world to "grey goo". But nothing like that will ever happen. The world is already awash in nanobots. We call them…
The danger of nanotechnology is that it can be built better than biological life. It could outcompete it in it's own environment, or at least different environments.
Solar panels can collect more energy than photosynthesis. Planes can fly faster than any bird. Guns are far more effective than any animal's weapons. Steam engines can run more efficiently than biological digestion. And we can get power from fuel sources biology doesn't touch, like fossil fuels or nuclear.
We conquered the macro world before we even invented electricity. Now we are just starting to conquer the micro world.
But AI is far more dangerous. It would take many many decades - perhaps centuries - of work to advance to that level. It's probably possible to build grey goo, it's just not easy or near. However AI could be much closer given the rapid rate of progress.
If you make an unfriendly AI, you can't just shut it off. It could spread it's source code through the internet. And it won't tell you that it's dangerous. It will pretend to be benevolent until it no longer needs you.
Re: Introducing OpenAI
#269> Musk: I think the best defense against the misuse of AI is to empower as many people as possible to have AI. If everyone has AI powers, then there’s not any one person or a small set of individuals who can have AI superpower. In a sense, we have no other defense. AI is just math and code, and I know of no way to distinguish good linear algebra from evil linear algebra. The barriers to putting that math and code tog…
Two people in a basement might make significant advances in AI research. So from the start, AI appears to be impossible to regulate. If an AGI is possible, then it is inevitable. Not necessarily. AGI might be possible but it's not necessarily possible for two people in a basement. AGI might require some exotic computer architecture which hasn't been invented yet, for example. This would put it a lot closer to nuclear…
Computers are far more general purpose. Any computer can theoretically run any program, it would just be slower, at worst.
Developing specialized hardware isn't out of reach, because of FPGAs.
Re: Introducing OpenAI
#270Earlier quoted context omitted.
And I think that you and the great-grandparent missed the point: It's not people, it's sentience that is dangerous. Since the dawn of time that meant man or gods. Soon that list will need to include computers.
Until that "soon" arrives, there's a far greater chance that people will wipe out humanity. We have more suicidal terrorists today than ever in history.
Suicidal terrorists are hardly a threat to the existence of humanity. It takes quite a bit more than explosive vests to destroy all of mankind.