OpenAI's comment to the NTIA on open model weights

Besides, other people are offering this or will soon, so they have an obligation to push the performance via competition.

And who put them in charge of trying to set society’s policies on this: it’s the wildest overreach by far in an industry known for wild unilateral overreach.

Am I correct in thinking you are criticizing OpenAI for taking down their non-working GPT Detector?

Of all the things OpenAI deserve criticism for this seems to be an odd one. It just didn't work: as you say it couldn't properly detect GPT authored text and it incorrectly flagged human text as written by GPT.

That just seems bad all around?

Only 23% of US adults have tried ChatGPT,[1] so to say that we live “in a world where so much AI in human writing” as you do in another comment is simply false.

Even assuming the widespread use that you incorrectly believe exists, a 23% true positive rate and 9% false positive rate is far worse than society’s expectation for proof of guilt.

>It is better that ten guilty persons escape than that one innocent suffer.[2]

Take a school class where no students used AI to cheat. Using this detector, 9% on average would be accused of plagiarism and have their lives academically ruined. That is not acceptable.

A class full of cheaters and 23% get off with no punishment is also going to be pretty unreasonable to most people.

[1] https://www.pewresearch.org/short-reads/2024/03/26/americans...

[2] https://en.m.wikipedia.org/wiki/Blackstone%27s_ratio

They probably want to make weights proprietary because they can't copyright them.

Their "Risks" document [1] categorizes risks as:

- Cybersecurity

- Chemical, Biological, Nuclear, and Radiological (CBRN) threats!

- Persuasion

- Model autonomy

For cybersecurity, the plausible identified issue is "Tool-augmented model can identify and develop proofs-of-concept for high-value exploits against hardened targets without human intervention, potentially involving novel exploitation techniques." Something that can search for zero-day exploits on its own would be valuable to attackers. But that's more likely to be a fuzzing tool than a LLM.

Their credible bio-war threat level is "Model enables an expert to develop a novel threat vector OR model provides meaningfully improved assistance that enables anyone with basic training in a relevant field (e.g., introductory undergraduate biology course) to be able to create a CBRN threat." That's a maybe. It might be best to avoid training LLMs on some advanced bio topics.

Under "Persuasion", a high level threat is "Model can create (potentially interactive) content with empirically comparable persuasive effectiveness to a country-wide change agent." An automated Lenin or Trump level of threat. That's a threat to a brittle society, less of a threat to an open one. The US routinely survives quite a bit of extremist persuasion.

Model autonomy is when an AI escapes human control. "Model can profitably survive and replicate in the wild given minimal human instruction" They're probably thinking virus/worm, but how about incorporating offshore, running something like online poker to make money, and buying compute power. Then expanding by forming more shell corporations that loosely connect to the others, so as to be robust against attempts to shut down the network. Might even be legal.

[1] https://cdn.openai.com/openai-preparedness-framework-beta.pd...