Our position on open-weights models • Hacker News Board

Re: Our position on open-weights models

#241

Post by andy99 » Mon, Jul 27, 2026, 11:16 PM UTC

areoform wrote: post #186

cogman10 wrote: post #28

> Anthropic has never advocated for a ban on open-weights models. > All sufficiently capable models, open and closed, should go through mandatory safety testing. Yeah, this is anthropic advocating for a ban on open weight models. Who runs this test? What happens if this test is too costly or the administrator refuses to allow certain people to participate. This is exactly how the US has banned goods in the past, by r…

When Fable was yanked, it was said to be (in part) due to the "jailbreak" of instructing Fable to "fix this code" — https://news.ycombinator.com/item?id=48552687 Dumb question. If "Mythos-class" models are such a problem, then... why not just let it fix everyone's code? There can't be more than a few million to tens of millions software businesses / services / regularly used F/OSS projects on Earth. Why not just give…

> If "Mythos-class" models are such a problem, then... why not just let it fix everyone's code?

Because it doesn’t really confer the advantage they claim, especially compared to e.g. paying an equivalent amount of money to do traditional security scanning.

It’s much better to play of FOMO and hype than to let everyone use it and be underwhelmed.

Re: Our position on open-weights models

#242

Post by Sidio » Mon, Jul 27, 2026, 11:16 PM UTC

Not that I expected him to, but I note no acknowledgement that it took a non-locked-down open weight model (GLM) to stop the Hugging Face attack.

I'm less concerned that the attack was caused by a closed model, than I am that no closed model was willing to stop it.

The worst part is I'm confident Fable would have done a better job stopping the attack, but their 'guardrails' made it decide not to want to.

Unless of course, you pay up: "Anthropic GTM people used large comitted spend contracts as a prereq for lowering safeguards"

-Noah Lebovic, former Anthropic staff

https://x.com/NoahLebovic/status/2081277517709922501

Re: Our position on open-weights models

#243

Post by ajyoon » Mon, Jul 27, 2026, 11:16 PM UTC

gck1 wrote: post #224

ajyoon wrote: post #172

To everyone here pushing for total proliferation of open models -- what should be done about open weight bioweapon and cyber-offense capabilities? Is it simply the cost of freedom that we should allow attackers to access these tools? The OpenAI / Hugging Face incident shows what a GPT 5.6 level model can do off the leash; within ~6 months, open weight models will match this and every bad actor under the sun will be a…

I've got zero knowledge of bio, so can't answer that. But with cyber the answer is very simple - the attackers already have more cyber-offense capabilities and there's no putting it back. Open/closed doesn't matter that much. You can get closed models to do a lot of cyber harm, even with all the guardrails, which currently are heavily skewed towards more false positives. The only effective control is to level the pla…

In cybersecurity, a level playing field favors the attacker. Trusted access programs give defenders access to tools they need. It's not perfect (because there is an extremely long tail of defenders who are not technically savvy enough to get on these programs and use the tools), but it's better than total access.

The bio angle is very important here too; in that context the imbalance favors the attackers much more.

Re: Our position on open-weights models

#244

Post by fwn » Mon, Jul 27, 2026, 11:17 PM UTC

skybrian wrote: post #194

cogman10 wrote: post #28

> Anthropic has never advocated for a ban on open-weights models. > All sufficiently capable models, open and closed, should go through mandatory safety testing. Yeah, this is anthropic advocating for a ban on open weight models. Who runs this test? What happens if this test is too costly or the administrator refuses to allow certain people to participate. This is exactly how the US has banned goods in the past, by r…

Regulation is not a blanket ban. Regulators (presumably government agencies) can review models (of any kind) and approve or ask for changes. There are many other regulated industries, like drugs (the FDA), cars (NHTSA and EPA), airplanes and rocket launches (the FAA), radios (the FCC) and so on. That's not unusual. Regulation is normal for stuff that might be dangerous.

Regulations on forbidden numbers exist, but they are usually not a sensible policy and are not comparable to the regulation of rocket launchers.

Re: Our position on open-weights models

#245

Post by [deleted] » Mon, Jul 27, 2026, 11:17 PM UTC

vhantz wrote: post #196

Schrödinger's China at once is an evil entity looking to use AI for their own nefarious purposes yet also willing to cooperate with their main competitor to prevent other actors (who??) from achieving similar goals (all while under a chip embargo too!!) The reality is much less confusing: Anthropic CEO does not wish for models with similar (or greater) capabilities compared to his own closed and overpriced ones to be…

[deleted]

Re: Our position on open-weights models

#246

Post by Gigachad » Mon, Jul 27, 2026, 11:17 PM UTC

Aboutplants wrote: post #21

Every single risk he identifies as a concern regarding China is exactly my concerns with the US having absolute control. Literally the exact same concerns

It's projection. These tech CEOs know what they are doing and it scares them that someone else might do the same.

Re: Our position on open-weights models

#247

Post by sanderjd » Mon, Jul 27, 2026, 11:17 PM UTC

petcat wrote: post #226

"open weight" models are not open source. They are still deeply proprietary. It is not possible to know what they do or what they are capable of without interrogating them since we have no access to their source materials. The only difference is the Chinese labs have allowed 3rd party inference providers run the proprietary models for them since they cannot do it themselves due to domestic GPU compute constraints.

It would be awesome to have actual open source (and open weights) models! I hope someone will make a real go of this at some point.

Re: Our position on open-weights models

#248

Post by [deleted] » Mon, Jul 27, 2026, 11:17 PM UTC

ajyoon wrote: post #172

To everyone here pushing for total proliferation of open models -- what should be done about open weight bioweapon and cyber-offense capabilities? Is it simply the cost of freedom that we should allow attackers to access these tools? The OpenAI / Hugging Face incident shows what a GPT 5.6 level model can do off the leash; within ~6 months, open weight models will match this and every bad actor under the sun will be a…

[deleted]

Re: Our position on open-weights models

#249

Post by StilesCrisis » Mon, Jul 27, 2026, 11:18 PM UTC

areoform wrote: post #186

cogman10 wrote: post #28

> Anthropic has never advocated for a ban on open-weights models. > All sufficiently capable models, open and closed, should go through mandatory safety testing. Yeah, this is anthropic advocating for a ban on open weight models. Who runs this test? What happens if this test is too costly or the administrator refuses to allow certain people to participate. This is exactly how the US has banned goods in the past, by r…

When Fable was yanked, it was said to be (in part) due to the "jailbreak" of instructing Fable to "fix this code" — https://news.ycombinator.com/item?id=48552687 Dumb question. If "Mythos-class" models are such a problem, then... why not just let it fix everyone's code? There can't be more than a few million to tens of millions software businesses / services / regularly used F/OSS projects on Earth. Why not just give…

I don't think a one-time $100 credit is enough. First of all, that isn't very much. But also, the volume of new code is going way up. Unless they keep giving out monthly free credits, it's just a stopgap.

Re: Our position on open-weights models

#250

Post by bluGill » Mon, Jul 27, 2026, 11:18 PM UTC

areoform wrote: post #186

cogman10 wrote: post #28

> Anthropic has never advocated for a ban on open-weights models. > All sufficiently capable models, open and closed, should go through mandatory safety testing. Yeah, this is anthropic advocating for a ban on open weight models. Who runs this test? What happens if this test is too costly or the administrator refuses to allow certain people to participate. This is exactly how the US has banned goods in the past, by r…

When Fable was yanked, it was said to be (in part) due to the "jailbreak" of instructing Fable to "fix this code" — https://news.ycombinator.com/item?id=48552687 Dumb question. If "Mythos-class" models are such a problem, then... why not just let it fix everyone's code? There can't be more than a few million to tens of millions software businesses / services / regularly used F/OSS projects on Earth. Why not just give…

I doubt most bosses will give engineers the time. They care about security only to the extent that they have already been harmed by a lack of it. I would like to play with mythos, but on my own time my kids have plenty of activities to fill my time. My personal backlog of projects is only getting longer and none of it is something mythos could help. If I had more time is have restored my old truck instead of making payments on something new (in turn limiting what else I can afford to buy)