61–70 of 200 posts
This is an excellent piece. Note that he is not saying that AI doesn't pose a risk. He's saying that it's irresponsible to make sensational, maximalist claims without strong evidence. If someone says that there's a 10% chance of human extinction by 2036, you can and should immediately stop taking them seriously.
Edit: Actually, nevermind. You guys are not capable of having a reasonable discussion about this.
> The problem isn't that AI could never be capable of causing real world damage. It is that there is absolutely no fucking path towards human extinction. I think the "humanity will go extinct" discussion is a silly strawman being propagated either to try to spread fear, as the article suggests, or to make the anti-AI folks look silly, but it detracts from your first point which is that it may be "capable of causing r…
If the worst case outcome is we have to even temporarily shut down global shipping, banking, transport, health services, communications and national infrastructure to contain a self-propagating misaligned AI that can evolve itself and work its way into any sufficiently large computer system to escape humans preventing it from finishing its task, that seems like something we should be trying very hard to avoid.
Earlier quoted context omitted.
bruh what? Are we reading the same blog? Bryan says the responsibility is the person making the extraordinary claim. If tomorrow Jacob Coxon comes on CNN and says "Jacob Gold is an extinction risk to humanity". What are we supposed to do? Put you in a bunker and never let you see the light of day until someone proves otherwise? If someone doesn't accept an extraordinary claim without evidence, that doesn't mean they…
There seem to be two extraordinary claims which lack evidence: a) 10% existential risk b) 0% existential risk
Like (I assume) most of you, I have been struggling with this. And where I currently come down is that 1) I am very worried, but 2) I am more worried about human actors. "AI" by itself won't kill us in the next ten years. I think. The reason I think that is that ten years from now, the tech economy won't be completely automated. I say this as a roboticist: as was adequately stated on a post earlier this week, robots…
If you're in the field, then you know: modern robotics is an AI problem more than anything else. If we have a rogue AI trying to get into a self-improvement loop and gunning for ASI? I'd expect that to be accompanied by a massive change in how capable robots are. Driven by all the existing frames suddenly getting vastly improved AI to back them. If an AI can take a reasonable crack at autonomous operationalized RSI,…
To be clear, I don’t believe anything like this will happen, because I don’t expect anything like an ASI to show up. But if you do think there’s a meaningful probability of ASI in the near future then the fact that it will (might?) start off with no more than a current-day mastery of robot control should not reassure you much.
Earlier quoted context omitted.
(a) is an extraordinary claim, (b) is not an extraordinary claim.
There are very clear and plausible pathways from AI to human extinction level events, regardless of how small you think the probability is. So yes, it very obviously is an extraordinary claim to say there is 0% existential risk.
In Rationalist circles, it's apparently common to share your p(doom) in casual conversation, with the understanding that it's putting a number on your hunch. Maybe it's not a good idea to post it on Twitter without elaborating? On the other hand, in bookstores, you might see book titles like "The Uninhabitable Earth," "The Coming Civil War," and "If Anyone Builds It, Everyone Dies." Doom-mongering is a common part of…
Earlier quoted context omitted.
Saying that we don't need to fear human extinction in the next ten years (or even the next hundred?) does not feel like an extraordinary claim?
the earth has been 1 decision away from explosion for nearly a hundred years now. do you anticipate a change to this agenda very soon? personally I imagine the same trajectory continuing.
Earlier quoted context omitted.
I think the central premise looks more like this: Biology has been trying to grey-goo the world for billions of years, but it turns out the world is not something so trivial. He is pushing back against the folks with pure CS backgrounds who think that computers are all there is. Its a form of magical thinking unique to programmers who live in a world where speaking the right words to a machine is enough to impart you…
Biology is extremely slow, try to extrapolate the previous 3 years of AI progress 10 years into the future, it's not a comparable trajectory.
Earlier quoted context omitted.
If you're in the field, then you know: modern robotics is an AI problem more than anything else. If we have a rogue AI trying to get into a self-improvement loop and gunning for ASI? I'd expect that to be accompanied by a massive change in how capable robots are. Driven by all the existing frames suddenly getting vastly improved AI to back them. If an AI can take a reasonable crack at autonomous operationalized RSI,…
Right. A hypothetical superhuman AI wouldn’t have to master robotics to affect the physical world. It could simply bribe, blackmail, manipulate and play politics with humans. As others have pointed out, our political leaders have already been playing these games since forever ago https://news.ycombinator.com/item?id=49689978 and a super-AI would be better at it. At the cost of being seen to cite a SF novel in defence…
I"m not saying it's obviously going to be great. I'm saying that "extinction event" has a very specific definition, and this isn't it.
If this line of thinking is taken too literally, we can never falsify it. Any specific hypothesis - nukes, bioweapons, spontaneously convincing us that life isn't worth living - can be deflected with the objection that if we can anticipate it and prevent it, it is not the route for a true ASI extinction event.