Live data from Hacker News

Our position on open-weights models

anthropic.com

291–300 of 1001 posts

Re: Our position on open-weights models

#291

> All sufficiently capable models, open and closed, should go through mandatory safety testing What happens if a model fails the test? Surely one can use Kimi K3 for evil, somehow or other. What now? "Mandatory safety testing" implies consequences for failing, yet Dario has nothing to say about what the consequences should be. He says he doesn't advocate a ban but it's hard to imagine what his alternative would be if…

If a model fails the test, it should be banned. He is not advocating a ban of open-weight models. He is advocating a ban of models that fail mandatory safety testing. Seems reasonable and straightforward.

Any sufficiently capable open weights model would fail "safety" testing though, as any "safeguards" of the sort Anthropic likes could be removed. That's just another way of saying they want a ban on capable open source models which would contradict their earlier statement, or at least make it very misleading. It's hard to see how this post can be internally consistent without some hint from Dario about what he believes should happen to models that fail safety testing and/or how capable open weights models could possibly pass a safety test of the kind he proposes.

Re: Our position on open-weights models

#292

> Anthropic has never advocated for a ban on open-weights models. > All sufficiently capable models, open and closed, should go through mandatory safety testing. Yeah, this is anthropic advocating for a ban on open weight models. Who runs this test? What happens if this test is too costly or the administrator refuses to allow certain people to participate. This is exactly how the US has banned goods in the past, by r…

This is my read too- if American companies start backing nonsense like this, they'll fall behind permanently.

> My primary concern is the risk that authoritarian governments—not solely the Chinese Communist Party (CCP), although the CCP is clearly the most capable threat—

Isn't this article an argument in favor of authoritarianism? Plus a tad hypocritical no? The US is on an obvious authoritarian path; complete with threatening their neighbors, murdering innocent civilians, and locking up innocent people in droves

Please stop giving this company money, people.

Re: Our position on open-weights models

#293

> All sufficiently capable models, open and closed, should go through mandatory safety testing What happens if a model fails the test? Surely one can use Kimi K3 for evil, somehow or other. What now? "Mandatory safety testing" implies consequences for failing, yet Dario has nothing to say about what the consequences should be. He says he doesn't advocate a ban but it's hard to imagine what his alternative would be if…

If a model fails the test, it should be banned. He is not advocating a ban of open-weight models. He is advocating a ban of models that fail mandatory safety testing. Seems reasonable and straightforward.

Can you think of anything open-source that has to go through mandatory tests to be distributed and still survived ?

There is a reason to it, that's as good as any angle to find why IMHO.

Re: Our position on open-weights models

#294
Dario thinks of policy as if the Berlin Wall fell yesterday, he is so detached from the reality of the world.

The way the world economy is right now with coercion being the norm between countries, there cannot be a global body for anything, certainly not one that is based here in the US.

Re: Our position on open-weights models

#295
post #265
post #237

It is obviously not going to be until some really bad series of cyberattacks or a chemical/bioweapon attack before anyone takes regulation of models seriously. They are _obviously_ (please convince me otherwise) going to be capable of carrying these terrible things out almost completely autonomously at some point in the near future, in potentially clever ways. Therefore we must, at some point, ban or heavily regulate…

Why would you assume that the same capability could not be used to harden systems?

Addressed in the source, but there's an asymmetry favoring the attackers. They only have to find one exploit once. The defenders have to be perfect all the time.

Re: Our position on open-weights models

#296
post #207

Someone who until yesterday did not seem bothered by his technology being possibly used to bomb elementary girls school in another country seems to suddenly care about the repression of citizens in yet another country. No, we don't buy your virtue signaling. And we certainly don't need your better-than-thou opinions on this year's "nightmare scenarios".

Yeah Dario is just flat out disgusting in term of how shamelessly hypocritical he is. Do people actually believe that he gives a shit about the well being of the Chinese people? If the U.S. starts a war with China start bombing Chinese cities Dario would absolutely jump onboard supporting it. He'd probably make Claude to add DeepSeek and Moonshot HQ to the targeting list lmao. He is super pro-Israel as well, and neve…

Please don't just copy paste existing comments

Re: Our position on open-weights models

#297
post #172

To everyone here pushing for total proliferation of open models -- what should be done about open weight bioweapon and cyber-offense capabilities? Is it simply the cost of freedom that we should allow attackers to access these tools? The OpenAI / Hugging Face incident shows what a GPT 5.6 level model can do off the leash; within ~6 months, open weight models will match this and every bad actor under the sun will be a…

There is also a whole second category of immense risks of having US companies gatekeeping offensive capabilities, especially for us here in Europe. The centralization/privacy/kill-switch concerns that come with it are a huge AI safety dimension.

I'd rather have a level playing field within a phase of adaptation and hardening regarding cybersecurity issues than a constant dependency on the US, maybe grabbing Greenland today, maybe "extracting" our president tomorrow.

The delta between privileged capabilities and open weight capabilities alone already is a massive, unaddressed AI safety risk.

Re: Our position on open-weights models

#298
> Anthropic has never advocated for a ban on open-weights models.

What are the legal ramifications of this statement if it turns out Anthropic have lobbied for this? Does it just get swept under the rug? I can't say this is bullshit (that would be defamatory) but I am intensely skeptical.

> China has limited domestic production capacity, and therefore, due to the scaling laws, cannot build more powerful models than the US without US chips.

This is playing to readers' biases; isn't DeepSeek V4 Pro deployed on Huawei Ascend already? The old "Chinese can only copy" meme is getting pretty tired these days.

> All sufficiently capable models, open and closed, should go through mandatory safety testing

Applying such standards in the US means that US defenders are blocked from using the models, but attackers from other countries aren't. That is clearly counterproductive.

It's already been pointed out quite eloquently elsewhere that there is no such thing as a safety filter because the LLM and external filters can't actually identify malicious use. They can only identify the weaker implication "if the user is malicious, this is bad."

Re: Our position on open-weights models

#299
What would it mean to crack down on distillation? I could think of a few possibilities:

1. Using political pressure to target companies that are accused of doing it.

2. Attempting to impose criminal penalties on individuals associated with the action.

3. Having the US government attempt to use its capabilities to stop it.

None of these seem particularly likely to succeed.

Re: Our position on open-weights models

#300

> Anthropic has never advocated for a ban on open-weights models. > All sufficiently capable models, open and closed, should go through mandatory safety testing. Yeah, this is anthropic advocating for a ban on open weight models. Who runs this test? What happens if this test is too costly or the administrator refuses to allow certain people to participate. This is exactly how the US has banned goods in the past, by r…

Yeah, seems pretty likely. Anthropic will make the case that their models should be evaluated with the safety layer in front, because that is the only way the model is available whereas open weight models need to pass the same test just on the weights. The economic implications will be rather large, but in terms of security it seems inconsequential. The most compelling argument would be that by limiting the use of op…

Just sell us the gate and we can run any open-source model behind it.
Post reply on HN