Live data from Hacker News

Our Agreement with the Department of War

openai.com

111–120 of 306 posts

Re: Our Agreement with the Department of War

#111
post #5

Not great? Seems kind of loose language? It isn't OpenAI saying no autonomous weapons use, but only that use must be consistent with laws, regulations, and department policies: "The Department of War may use the AI System for all lawful purposes, consistent with applicable law, operational requirements, and well-established safety and oversight protocols. The AI System will not be used to independently direct autonom…

[deleted]

Re: Our Agreement with the Department of War

#112

I don't think Anthropic is a saint that will never do anything unethical. I don't think ChatGPT is any better or worse. But I do think my cancelling ChatGPT so I can try Claude, at this time, sends the message I want to send, which is why I did it.

And a nice bonus is that Claude is way better than ChatGPT right now anyway

How so, it’s unstable like floating ice.

Re: Our Agreement with the Department of War

#113
post #10

> The Department of War may use the AI System for all lawful purposes, consistent with applicable law, operational requirements, and well-established safety and oversight protocols. The AI System will not be used to independently direct autonomous weapons in any case where law, regulation, or Department policy requires human control , nor will it be used to assume other high-stakes decisions that require approval by…

Didn't fully follow the saga, but isn't their "imposing their own morals" is that "we do not want to allow you to let our AI go on an unsupervised killing spree"?

Re: Our Agreement with the Department of War

#114
post #10

> The Department of War may use the AI System for all lawful purposes, consistent with applicable law, operational requirements, and well-established safety and oversight protocols. The AI System will not be used to independently direct autonomous weapons in any case where law, regulation, or Department policy requires human control , nor will it be used to assume other high-stakes decisions that require approval by…

My interpretation of the difference is more like: Anthropic wanted the synchronous real-time authority to say "No we wont do that" (e.g. by modifying system prompts, training data, Anthropic people in the loop with shutdown authority). OpenAI instead asked for the asynchronous authority to re-evaluate the contract if it is breached (e.g. the DoD can use OpenAI tech for domestic surveillance, but there's a path to contract and service termination if they do this).

If my read is correct: I personally agree with the DoD that Anthropic's demands were not something any military should agree to. However, as you say, the DoD's reaction to Anthropic's terms is wildly inappropriate and materially harmed our military by forcing all private companies to re-evaluate whether selling to the military is a good idea going forward.

The DoD likely spends somewhere on the order of ~$100M/year with Google; but Google owns a 14% stake in Anthropic, who spends at least that much if not more on training and inference. All-in-all, that relationship is worth on the order of ~$10B+. If Google is put into the position of having to decide between servicing DoD contracts or maintaining Anthropic as an investee and customer, its not trivially obvious that they'd pick the DoD unless forced to with behind-the-scenes threats and the DPA. Amazon is in a similar situation; its only Microsoft that has contracts large enough with the DoD where their decision is obvious. Hegseth's decision leaves the DoD, our military, and our defense materially weaker by both refusing federal access to state of the art technology, and creating a schism in the broader tech ecosystem where many players will now refuse to engage with the government.

Either party could have walked away from negotiations if they were unhappy with the terms. Alternatively: the DoD should have agreed to Anthropic's red lines, then constrained/compartmentalized their usage of Anthropic's technology to a clearly limited and non-combat capacity until re-negotiation and expansion of the deal could happen. Instead, we get where we're at, which is not good.

IMO: I know a lot of people are scared of a fascist-like future for the US, but personally I'm more fearful of a different outcome. Our government and military has lost all of its capacity to manufacture and innovate. Its been conceded to private industry, and its at the point where private industry has grown so large that companies can seriously say "ok, we won't work with you, bye" and it just be, like, fine for their bottom line. The US cannot grow federal spending and cannot find a reasonable path to taxing or otherwise slowing down the rise of private industry. We're not headed into fascism (though there are elements of that in the current admin): We're headed into Snow Crash. The military is just a thin coordination layer of operators piecing together technology from OpenAI, Boeing, Anduril, Raytheon. Public governments everywhere are being out-competed by private industry, and in some countries it feels like industry tolerates the government, because it still has some decreasing semblance of authority, but especially in the US that semblance of authority has been on a downward trend for years. Google's revenue was 7% of the US Federal Government's revenue last year. That's fucking insane. What happens when we get to the point where Federal debt becomes unserviceable? When Google or Apple or Microsoft hit 10%, or 15%? Our government loses its ability to actually function effectively; and private industry will be there to fill the void.

Re: Our Agreement with the Department of War

#117
> For intelligence activities, any handling of private information will comply with the Fourth Amendment, the National Security Act of 1947 and the Foreign Intelligence and Surveillance Act of 1978, Executive Order 12333, and applicable DoD directives requiring a defined foreign intelligence purpose. The AI System shall not be used for unconstrained monitoring of U.S. persons’ private information as consistent with these authorities. The system shall also not be used for domestic law-enforcement activities except as permitted by the Posse Comitatus Act and other applicable law.

My reading of this is that OpenAI's contract with the Pentagon only prohibits mass surveillance of US citizens to the extent that that surveillance is already prohibited by law. For example, I believe this implies that the DoW can procure data on US citizens en masse from private companies - including, e.g., granular location and financial transaction data - and apply OpenAI's tools to that data to surveil and otherwise target US citizens at scale. As I understand it, this was not the case with Anthropic's contract.

If I'm right, this is abhorrent. However, I've already jumped to a lot of incorrect conclusions in the last few days, so I'm doing my best to withhold judgment for now, and holding out hope for a plausible competing explanation.

(Disclosure, I'm a former OpenAI employee and current shareholder.)

Re: Our Agreement with the Department of War

#118

> For intelligence activities, any handling of private information will comply with the Fourth Amendment, the National Security Act of 1947 and the Foreign Intelligence and Surveillance Act of 1978, Executive Order 12333, and applicable DoD directives requiring a defined foreign intelligence purpose. The AI System shall not be used for unconstrained monitoring of U.S. persons’ private information as consistent with t…

thanks for speaking out, and yes, that was my interpretation, as well, which I outlined below. This is nothing more than some sugar coating on "lawful use" despite what OpenAI says and the contractual "safeguards" they tout like the FDEs.

Re: Our Agreement with the Department of War

#119
post #46

Earlier quoted context omitted.

> OpenAI acceded to demands that the US Government can do whatever it wants that is legal. Anthropic wanted to impose its own morals into the use of its products. Excuse me, but what a fucked up perspective. "Impose its own morals into the use of its products"? What happened to "We give each other the freedom to hold beliefs and act accordingly unless it does harm"? How on earth did it come to something where the fra…

Nobody is saying that Anthropic has to shut down. They’re just saying that nobody taking government money can pay Anthropic for their service as a part of that contract. Anthropic still has the right to exist on their own terms, but their business model is based on rapidly-increasing enterprise subscriptions, which included public sector spending. If Anthropic can survive on open source contributors shelling out $200…

Well it depends. Being that the federal government constitutes 20% of the US economy, telling federal agencies you cannot contract with someone because they are adversarial to the USA is indeed pretty severe. When in reality they are not adversarial. We have no choice but to pay taxes and make the federal government 20 percent of our economy. There is no single company or any other entity that is close. And extending it to everyone who has a government contract probably makes it the majority of the economy. So it is not at all equivalent to a private company making a choice

Re: Our Agreement with the Department of War

#120

Earlier quoted context omitted.

The GP's use of the word "impose" didn't seem perjorative to me or suggest that Anthropic is the offender and the government is the victim. I think you're reading a lot into a simple word choice and this response seems way too hostile.

Are you really going to pretend that “impose their morals” is a completely value-neutral statement?

It certainly was intended as such. In a commercial transaction, that's what they're doing. They don't think it's moral to use their product in certain ways. They are thus prohibiting their customer from using it in such ways.

But, as I've said, I tend to agree with both Anthropic and the Administration's positions. What was wrong here is that rather than just terminating the contract, the Administration went nuclear.

Post reply on HN