81–90 of 158 posts
Naive question: Isn't the easiest way to prevent an general purpose AI from taking over the world to not make a general purpose AI?
Naive question: Isn't the easiest way to prevent an general purpose AI from taking over the world to not make a general purpose AI?
Earlier quoted context omitted.
Pretty sure the self driving vehicles get the speed data from a dataset and not from signs.
then it's not really (self)driving.
Naive question: Isn't the easiest way to prevent an general purpose AI from taking over the world to not make a general purpose AI?
Unfortunately that just means you don’t create one. To prevent one being created, you have to either somehow figure out a way to get everyone in the world to agree not to create one, or obtain enough global power that you can forcibly stop anyone creating one. Not exactly easy!
Naive question: Isn't the easiest way to prevent an general purpose AI from taking over the world to not make a general purpose AI?
Best case, that AI can prevent the creation of harmful AI, though that's glossing over a lot of details that I'm not qualified to describe.
"When you start reading about AI safety, it’s striking how there are two separate communities—the one mostly worried about machine learning perpetuating racial and gender biases, and the one mostly worried about superhuman AI turning the planet into goo" - great quote.
It's not that I'm not concerned with bias and AI systems going haywire, but the above scenarios seem to get less attention from researchers, probably because their employers might be perpetuating many of these above issues of AI safety.
Seeing someone as trustworthy as Scott choose to work on AI safety is a pretty good sign for the state of the field IMO. It seems like a lot of studious people agree AI alignment is important but then end up shoehorning the problem into whatever framework they are most expert in. When all you have is a hammer etc... I feel like he has good enough taste to avoid this pitfall. Semi-related - I'd want to see some actual…
Earlier quoted context omitted.
Agreed. Maybe... Short sighted vs. far sighted vs. ignorance is bliss?
It's a secular/religious divide. (As it says in the post.) Though it's possible the people who think a theoretical future AI will turn the planet into paperclips have merely forgotten that perpetual motion machines aren't possible.
Part of such precautionary planning involves asking whether such an accident could happen easily or not. There certainly isn't consensus at the moment, but the philosophy very clearly favors a cautious approach.
Most people are used to thinking about established science that follows expected rules, or incremental advances that have no serious practical consequences. But this isn't that. There is good reason to think that we're approaching a step-change in capabilities to shape the world, and even a strong suspicion of this warrants taking serious defensive measures. Crucially for this particular instance of the discussion, OP is favoring that.
There will necessarily be a broad spectrum of opinions regarding how to handle this, both in the central judgement and how palatably the opinion itself is presented. Using a dismissive moniker like 'religious' for a whole segment of it doesn't give justice to the arguments.
Present a counterargument if you feel strongly about it, and see whether that will stand on its own merit.
Earlier quoted context omitted.
I think you are conflating different concepts. It is clearly imaginable that a very intelligent agent could end humanity if its objective would require so. How exactly is "perpetual motion machines can't exist" related to this? How is it going to prevent an agent from engineering 1000 pandemic viruses at the same time?
> It is clearly imaginable that a very intelligent agent could end humanity if its objective would require so. This is quite possible. Indeed, I don't believe this is exclusive to superintelligence or requires it at all. Compare to the closest thing we have to "inventing AGI" - having babies. People do that all the time and there isn't a mathematical guarantee that baby won't end humanity, but we don't do much to sto…
The anthropomorphism is misleading. No one expects that an AGI would "want to" in the commonplace sense of being motivated by animosity, fear, or desire. The problem is that the best path to satisying its reward function could have adverse-to-extinction level consequences for humanity, because alignment is hard, or maybe impossible.