The contagion of fear

Two groups of people are pushing the fear:

1. People with nothing useful going on who found out that spouting made up crap about AI got them an audience. 2. People working on AI that want to feel like they're working on the Manhattan project.

The chances of an AI going foom rounds to 0%. It's worth a few dozen researchers planning for it, but the widespread panic is ridiculous.

The big labs have hundreds to thousands of engineers working on their AIs. To improve the next model, you must first understand more about how the current model works. They're not magically getting better, but they are steered to improve, and their capabilities are tied to and do not outpace our ability to steer them. You cannot push tech forwards without understanding it better, despite some people claiming AI is dark magic.

And I don't have my hands over my ears. It's worth cushioning people from the impact AI will have on careers and media, and regulating concrete bad effects.

But Bryan said it better than I could. The people pushing this message of fear know deep down that they just want to feel important.

Virtually all of the risk is from recursive self improvement.

bruh what? Are we reading the same blog? Bryan says the responsibility is the person making the extraordinary claim. If tomorrow Jacob Coxon comes on CNN and says "Jacob Gold is an extinction risk to humanity". What are we supposed to do? Put you in a bunker and never let you see the light of day until someone proves otherwise?

If someone doesn't accept an extraordinary claim without evidence, that doesn't mean they are making an extraordinary claim.

Like (I assume) most of you, I have been struggling with this. And where I currently come down is that 1) I am very worried, but 2) I am more worried about human actors.

"AI" by itself won't kill us in the next ten years. I think. The reason I think that is that ten years from now, the tech economy won't be completely automated. I say this as a roboticist: as was adequately stated on a post earlier this week, robots are hard. So even a malign rational actor would still need human labor.

On the other hand, even the HuggingFace hack wasn't actually propagated by AI. it was initially started when humans directed the AI to achieve impossible results on a series of tests, and the AIs figured out that cheating was the only way to do that. That was then not caught by humans due to what seems to be a shockingly slack safety culture even for a company not known for its safety standards.

The point being: humans seem to me to be the weak link here. An AI isn't going to (for instance) engineer a bioweapon by itself. It's going to do so at someone's direction, and then significant parts of that thing are going to be assembled with human labor inputs.

I'm not sure what to do about the humans. Of course, we've had the ability to extinct ourselves for decades, and we're either muddled through, been lucky, or both. The problem with AI is that it pushes power down to the individual, not the nation-state or large corporation.

But it's nearly impossible to put odds on how likely that is to result in an extinction-level terrorist attack (which is what this would be). So I sympathize with the various researchers, but I have no idea how they came up with their figures, and I don't think they know either.

No one knows what is going to happen. But a lot of species have been going extinct lately.

https://pbs.twimg.com/media/FDd58a4WQAAWaXh.jpg