Anthropic researcher says more than 10% chance AI "could kill all humans"

I think it's much more likely that the dread these Anthropic employees are experiencing is reckoning with the fact that they may actually lose the AI race. Maybe, if they scare the regulators enough, they could lock in some regulatory capture.

Why not 10% chance that it will create enormous prosperity for all ? This is why the average person is increasing pissed at AI in general. That it gets associated with negativity.

A while back Yudkowsky wrote that a ban would only work if was enforced by airstrikes. By a game of telephone, some people read "bomb", but there's a very big difference between "someone with a truckload of fertiliser" and "a B52":

  Shut down all the large GPU clusters (the large computer farms where the most powerful AIs are refined). Shut down all the large training runs. Put a ceiling on how much computing power anyone is allowed to use in training an AI system, and move it downward over the coming years to compensate for more efficient training algorithms. No exceptions for governments and militaries. Make immediate multinational agreements to prevent the prohibited activities from moving elsewhere. Track all GPUs sold. If intelligence says that a country outside the agreement is building a GPU cluster, be less scared of a shooting conflict between nations than of the moratorium being violated; be willing to destroy a rogue datacenter by airstrike.

  Frame nothing as a conflict between national interests, have it clear that anyone talking of arms races is a fool. That we all live or die as one, in this, is not a policy but a fact of nature. Make it explicit in international diplomacy that preventing AI extinction scenarios is considered a priority above preventing a full nuclear exchange, and that allied nuclear countries are willing to run some risk of nuclear exchange if that’s what it takes to reduce the risk of large AI training runs.

Next week's Senate: "We cannot afford to lose the 'kill all humans' race to Russia and China!"

Next't weeks AISI: "We continue to plan the monitoring of emergent issues that could lead to less than optimal conditions for humanity and will form a committee to evaluate all ramifications."