> You can, in fact, opt out. You can opt out and do your damndest to stop what's happening, throw every cent you have at it, bend any ear that will listen, make use of the fact that your voice (as Anthropic leadership) has some meaningful weight.
There are billions of people who have opted out of playing the game. Has the game stopped? Has any game stopped because the people not playing it decided that it ought to? Only with government intervention, which is exactly what you just criticized Anthropic for being disingenuous for requesting.
Is your position that they should just be smart bloggers asking for regulation, instead of the preeminent lab asking for regulation, and that would be either more ethical or more effective? If it's less effective, isn't it de facto less ethical?
What say you about the thousands of smart bloggers asking for regulation who are ignored every single day and have no tools besides their blogs to steer the outcome?
> burn your position in the race to show just how fucking serious you are.
This is incredibly naive. Literally no one who is unconvinced of AI doom would be convinced by this... because they already don't believe the premise. Such a gesture would be readily explained away as "you were losing the race," or "you got rich enough already." This is the attitude when any individual opts out of participation (see: Hinton) and it's ridiculous to assume it'd be different if an entire company did it.
Not to mention, that an entire company can't do it. These companies have boards of directors. They are accountable to shareholders. A CEO who wanted to do this would simply be fired and the company would carry on. This is one of the key components of the trap. Large companies are not under the control of people but of incentives. They are literally deliberately designed not to be under the control of individuals –– to be immune to exactly the type of behavior you think is possible.
And yes, nuclear weapons are analogous to AI in the arms race dynamic to create and proliferate them. They are probably not analogous in there exists a stable equilibrium in nuclear weapons due to "accidents" of their nature. There need not be a similar equilibrium among competitive AIs.
----
And yes, your comment lands in exactly the category I mentioned. You do not believe the AI doom fears, so the behavior looks one way. I do believe in the AI doom concerns, so the behavior looks another way. This applies equally to yesterday's actions, today's actions, and tomorrow's, including some hypothetical honorable self-immolation to slow progress: I would see that as concordant with AI doom concerns, you would not. You would find it "hard not to be cynical" about the fact they already earned a ton of money, maybe was losing a bit of ground in the race, so on and so forth. This is plainly obvious to anyone who has had to converse with no-doomers, who can only analyze other people's behaviors under their own belief system, so it won't happen.
The only variable of disagreement is around AI doom.