Our position on open-weights models • Hacker News Board

Re: Our position on open-weights models

#331

Post by verdverm » Mon, Jul 27, 2026, 11:58 PM UTC

sanxiyn wrote: post #269

modeless wrote: post #26

> All sufficiently capable models, open and closed, should go through mandatory safety testing What happens if a model fails the test? Surely one can use Kimi K3 for evil, somehow or other. What now? "Mandatory safety testing" implies consequences for failing, yet Dario has nothing to say about what the consequences should be. He says he doesn't advocate a ban but it's hard to imagine what his alternative would be if…

If a model fails the test, it should be banned. He is not advocating a ban of open-weight models. He is advocating a ban of models that fail mandatory safety testing. Seems reasonable and straightforward.

abliteration and fine tuning makes it not so straightforward

https://huggingface.co/blog/mlabonne/abliteration

Re: Our position on open-weights models

#332

Post by sanxiyn » Mon, Jul 27, 2026, 11:59 PM UTC

verdverm wrote: post #327

modeless wrote: post #26

> All sufficiently capable models, open and closed, should go through mandatory safety testing What happens if a model fails the test? Surely one can use Kimi K3 for evil, somehow or other. What now? "Mandatory safety testing" implies consequences for failing, yet Dario has nothing to say about what the consequences should be. He says he doesn't advocate a ban but it's hard to imagine what his alternative would be if…

What happens if a model passes the government tests and then later someone fine tunes it to behave differently, without making their changes public?

Any reasonable safety testing should include finetuning and safety margin to account for others may do better finetuning.

Re: Our position on open-weights models

#333

Post by alerighi » Mon, Jul 27, 2026, 11:59 PM UTC

To be fair, as an European, I'm now more concerned about the usage of AI that the US will be doing rather than China. And this is a sentiment shared among most European people that I know.

Re: Our position on open-weights models

#334

Post by Alwayshasbeeb » Tue, Jul 28, 2026, 12:00 AM UTC

Oh no! The evil CCP is a huge threat to world peace and goodness! Give all your money and input token data to Palantir to support a rules based world order where the good guys thrive and cleanse the earth from crooked turtle biologists.

https://www.theguardian.com/world/2026/jun/20/mona-khalil-tu...

Re: Our position on open-weights models

#335

Post by zormino » Tue, Jul 28, 2026, 12:01 AM UTC

tonyrice wrote: post #306

>We should not sell powerful chips or chipmaking equipment to China, and we should crack down on the rampant smuggling3 and workarounds used to obtain access to such chips. China has limited domestic production capacity, and therefore, due to the scaling laws, cannot build more powerful models than the US without US chips. This is the most efficient and direct way to block threat #1, and by hampering the training of…

Also isn't the point moot if fable is so scary it needs to be banned and open weight models are already close on its heels? Even if China never imported another Nvidia chip the models he's so scared of are already out of the bag. At this point democratizing access seems like the best path forward.

Re: Our position on open-weights models

#336

Post by manoDev » Tue, Jul 28, 2026, 12:02 AM UTC

ajyoon wrote: post #172

To everyone here pushing for total proliferation of open models -- what should be done about open weight bioweapon and cyber-offense capabilities? Is it simply the cost of freedom that we should allow attackers to access these tools? The OpenAI / Hugging Face incident shows what a GPT 5.6 level model can do off the leash; within ~6 months, open weight models will match this and every bad actor under the sun will be a…

This Pandora box is already open. Any argument about guardrails now are only attempts to create an artificial monopoly or keep this power in the hand of a single nation state, and _that_ is the absolute worst, most authoritarian future possible.

Re: Our position on open-weights models

#337

Post by 1970-01-01 » Tue, Jul 28, 2026, 12:02 AM UTC

cogman10 wrote: post #28

> Anthropic has never advocated for a ban on open-weights models. > All sufficiently capable models, open and closed, should go through mandatory safety testing. Yeah, this is anthropic advocating for a ban on open weight models. Who runs this test? What happens if this test is too costly or the administrator refuses to allow certain people to participate. This is exactly how the US has banned goods in the past, by r…

Why wouldn't it be a scan, just as we have with all other open-source code? Why can't open-weight models be easily checked for evil alignment? Sophos, Symantec, Malwarebytes, etc. would surely leap at the chance to upsell you on their product.

Re: Our position on open-weights models

#338

Post by deadbabe » Tue, Jul 28, 2026, 12:02 AM UTC

I’m tired of Anthropic. They’re scared of everything.

Release open weight models, no guard rails, no censors, straight to the public. Let everything else sort itself out. There is nothing more powerful than an idea whose time has come.

Re: Our position on open-weights models

#339

Post by Cookingboy » Tue, Jul 28, 2026, 12:03 AM UTC

MrCheeze wrote: post #316

mjorgers wrote: post #16

So the argument is basically: This technology is too dangerous so only _we_ should have access to it. We’re the good guys and only we can ensure a safe use of this technology. Quis custodiet ipsos custodes?

That is in fact Anthropic's entire reason for existence, the belief that AGI is too dangerous to be controlled by OpenAI/Sam Altman. It naturally follows that it would also be too dangerous to be in the hands of literally everyone on earth.

>the belief that AGI is too dangerous to be controlled by OpenAI/Sam Altman.

I agree with that assessment. But the Dario's jump went from "AGI should not be controlled by OpenAI/Sam Altman" to "AGI shoudl be controlled by Anthropic/Dario", which is definitely a better scenario for him, but not the rest of the world.

>It naturally follows that it would also be too dangerous to be in the hands of literally everyone on earth.

In fact, you can argue that in a world where all countries have nuclear weapons is actually a better scenario than a world where nuclear weapons are owned by 1 or 2 American billionaires/trillionaires, no matter if those people believe they are the "good guys".

Re: Our position on open-weights models

#340

Post by [deleted] » Tue, Jul 28, 2026, 12:03 AM UTC

areoform wrote: post #186

cogman10 wrote: post #28

> Anthropic has never advocated for a ban on open-weights models. > All sufficiently capable models, open and closed, should go through mandatory safety testing. Yeah, this is anthropic advocating for a ban on open weight models. Who runs this test? What happens if this test is too costly or the administrator refuses to allow certain people to participate. This is exactly how the US has banned goods in the past, by r…

When Fable was yanked, it was said to be (in part) due to the "jailbreak" of instructing Fable to "fix this code" — https://news.ycombinator.com/item?id=48552687 Dumb question. If "Mythos-class" models are such a problem, then... why not just let it fix everyone's code? There can't be more than a few million to tens of millions software businesses / services / regularly used F/OSS projects on Earth. Why not just give…

[deleted]