The Pentagon threatens Anthropic

    These consequences are generally understood to be some mix of :
    
    canceling the contract
    
    using the Defense Production Act, a law which lets the Pentagon force companies to do things, to force Anthropic to agree.
    
    the nuclear option, designating Anthropic a “supply chain risk”. This would ban US companies that use Anthropic products from doing business with the military2. Since many companies do some business with the government, this would lock Anthropic out of large parts of the corporate world and be potentially fatal to their business3. The “supply chain risk” designation has previously only been used for foreign companies like Huawei that we think are using their connections to spy on or implant malware in American infrastructure. Using it as a bargaining chip to threaten a domestic company in contract negotiations is unprecedented.

Look, you can't have a (working, democratic) government where one party can send the other to jail as soon as they get into power. If presidents could go to jail for doing their job, their opposing party would absolutely try to send them there.

This would then ultimately handicap the president: anything they do that the opposition can find a legal justification against could land them in jail, so they won't do anything that comes close to that. We do not want our chief executive making key decisions for the country based on fear of political retribution!

The Supreme Court has failed, miserably and repeatedly lately, and some of their decisions run directly counter to the law (often they even contradict past decisions!) But deciding the president won't face political retribution for trying to do his job was not a mistake.

We are creating a worse version of the Panopticon than was originally designed. A Panopticon that could have entirely devastating consequences. Not only is "the guard" able to see what any given "prisoner" is doing at any time, but they can look into the past. The self-regulation happens because the prisoners could be being watched. It is Orwellian. But this thing we're building? It can look at the prisoners' actions before it was even completed.

I think people don't think about this enough. Culture changes and in that time what is considered morally justifiable or even reasonable changes. Sometimes it is easy to judge people in the past by our current standards but other times it is not. Other times there is context needed, which is lost not only by time but in what is never recorded. How do prisoners self-regulate to future values that they do not know they are supposed to align to?

This creates a terrible machine where whoever controls it will likely have the power to prosecute anyone arbitrarily. Get the morals to change just slightly or just take things out of context and you have the public demanding prosecution. I think people think this seems far fetched but I'm willing to bet every single person on HN has fallen for some disinformation campaign. Be it the "carrots help you see in the dark", peoples misunderstanding between paper/plastic/canvas tote bags, a wide variety of topics related to environmentalism, and on and on. Even if you believe you have never fallen for such a disinformation (or malinformation) campaign, you'll have to concede that it is common for others to. That's all that is needed for someone in power to execute on this Panopticon, and it is a strategy people with power have been refining for thousands of years.

I really do support Anthropic pushing back here, but the discussions about "Future Claude" really are unsettling. It is like we are treating this as an inevitability. As if we have no choice in the matter. If that is true, then we are the mindless automata and then what does the military need killer-bots for? The would already have them.

[0] https://en.wikipedia.org/wiki/Panopticon