Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Our position on open-weights models

anthropic.com

Post reply on HN
Print view View original

481–490 of 1001 posts

Re: Our position on open-weights models

#481

Post by protocolture »

coffeemug wrote: post #176
cogman10 wrote: post #28

> Anthropic has never advocated for a ban on open-weights models. > All sufficiently capable models, open and closed, should go through mandatory safety testing. Yeah, this is anthropic advocating for a ban on open weight models. Who runs this test? What happens if this test is too costly or the administrator refuses to allow certain people to participate. This is exactly how the US has banned goods in the past, by r…

A government agency tests all medications, why not models?

Actually someone needs to make the positive case sufficiently well first.

Re: Our position on open-weights models

#482

Post by protocolture »

teravor wrote: post #85
pcstl wrote: post #38

[flagged]

> The NSA and the CIA with the same models, on the other hand, would use them exclusively for the good of the common man. has anyone ever made this absurd argument? the real argument is that CCP will leverage AI against US interests, which is obvious. it's weird how so many people pretend that they are citizens of the world and above it all.

[flagged]

Re: Our position on open-weights models

#483

Post by lenerdenator »

Cookingboy wrote: post #238
m3h wrote: post #207

Someone who until yesterday did not seem bothered by his technology being possibly used to bomb elementary girls school in another country seems to suddenly care about the repression of citizens in yet another country. No, we don't buy your virtue signaling. And we certainly don't need your better-than-thou opinions on this year's "nightmare scenarios".

Yeah Dario is just flat out disgusting in term of how shamelessly hypocritical he is. Do people actually believe that he gives a shit about the well being of the Chinese people? If the U.S. starts a war with China start bombing Chinese cities Dario would absolutely jump onboard supporting it. He'd probably make Claude to add DeepSeek and Moonshot HQ to the targeting list lmao. He is super pro-Israel as well, and neve…

> Meanwhile the world's most repressive government is North Korea, and obviously they don't even need AI to achieve that.

The country that exists solely because of China? That North Korea?

I mean I get your point, all of these guys are elitist authoritarians who will do anything for a buck, but I wouldn't bring up the DPRK in this discussion.

Re: Our position on open-weights models

#485

Post by girfan »

If nothing else, Anthropic has explicitly acknowledged the threat to its moat and that the margin is thin between its models and open-weight models. I'd take that as a win for open-weight models.

Good luck preventing distillation and limiting the supply of accelerators to China when Jensen himself is a strong opponent of any such barriers [1].

[1] https://youtu.be/Hrbq66XqtCo?si=VJh2QjOzkqT3iMjl&t=3456

Re: Our position on open-weights models

#486

Post by pianopatrick »

jackdeansmith wrote: post #450
pianopatrick wrote: post #116

I don't really see the link between use of AI and military superiority. My current understanding is a lot of current US military problems are due to rare earths supply chains. I don't see how AI would either help or hurt with that.

I see a very clear link, very capable AI models are clearly able to find and exploit cyber vulnerabilities at a speed and scale not possible before. In the right hands, that's clearly a weapon

I don't know how much of an edge hacking skills really give in war these days. A lot of America's adversaries seem to have learned tactics to deal with getting hacked. For example, Russia has these drones that use fiber optic cable so there is no way to jam or hack them.

Re: Our position on open-weights models

#487

Post by consumer451 »

tacet wrote: post #476
ajyoon wrote: post #172

To everyone here pushing for total proliferation of open models -- what should be done about open weight bioweapon and cyber-offense capabilities? Is it simply the cost of freedom that we should allow attackers to access these tools? The OpenAI / Hugging Face incident shows what a GPT 5.6 level model can do off the leash; within ~6 months, open weight models will match this and every bad actor under the sun will be a…

the bioweapon panic is funny. "oh, yes i know nothing about bicrobiology but i will follow instructions of synthetic text generation machine on temperature 1 about how to design a lab to not kill myself while brewing organisms that will kill myself if i make mistake" There is nothing that special about bioweapons, there are plenty of bacteria that will kill you just fine. Americans even have free samples on their sal…

> There is nothing that special about bioweapons.

The reason that madmen and terrorists choose kinetic weapons is because the knowledge and materials are more readily available... of and also that even terrorists are likely aware that their own people would suffer. As the knowledge and tools for playing with CRISPR-style biological legos become more widespread, we come closer to the Great Filter, where one person could kill billions.

Even our normal mad leaders have agreed that bioweapons cannot be allowed:

https://en.wikipedia.org/wiki/Biological_Weapons_Convention

Re: Our position on open-weights models

#488

Post by zmmmmm »

cogman10 wrote: post #28

> Anthropic has never advocated for a ban on open-weights models. > All sufficiently capable models, open and closed, should go through mandatory safety testing. Yeah, this is anthropic advocating for a ban on open weight models. Who runs this test? What happens if this test is too costly or the administrator refuses to allow certain people to participate. This is exactly how the US has banned goods in the past, by r…

we should turn this around

Private models should be banned because they can't be transparently evaluated. We have to trust the same entities that made them to evaluate them, in spite of their gigantic conflict of interest in doing so.

Therefore only open weight models can be allowed, since this allows genuine third party evaluation.

Re: Our position on open-weights models

#489

Post by credit_guy »

djsjajah wrote: post #422
credit_guy wrote: post #370

I know it's unpopular, or unfashionable, but I agree with this letter. LLMs are becoming so powerful that they are dangerous. We've seen last week with the OpenAI hacking (by mistake) Hugging Face debacle. It is absolutely ok to have open weight models at the level of GPT-OSS-100B. That one was released one year ago, and I think it's still a strong one. GLM 5.2 is a whole new level, but it appears to still be safe. M…

but what is the point? A ban is supposed to make a certain thing less likely to occur. Does a ban of open source models do that? Presumably, the behavior you are trying to limit is the miss-use of these models but I don't know how many state sponsored hacking groups are going to give a ban a second thought.

Yes, that's how I read Amodei's post.

Imagine Kimi K4 will be as powerful as Mythos. Anthropic can work for months and months to set up guardrails on Mythos, so when the model is finally released, it will generally decline to help hackers develop and prosecute cyberattacks, and if they do, at least there would be a trace so the law enforcement can track the perpetrators. Let's now say that Kimi K4 is released after a similar effort to develop guardrails. But being open weights, someone can just take the model, and finetune it until it does not refuse to assist in developing cyberattacks, and moreover, those people can run the model on their own private GPU cluster, so nobody can track the attack back to them. The situation is actually worse than that, most likely. Guardrails might be just markdown documents which are added to the context like regular skills. Then removing the guardrails for an open weights model does not even involve any finetuning, just removing some docs from a harness.

Re: Our position on open-weights models

#490

Post by teacpde »

soundworlds wrote: post #257

What Dario misses time and time again, is that people don't trust the US to create aligned AI anymore. His entire strategy rests on the assumption that the US (and their government) are exceptional. This is clearly false to the rest of the world.

He did not miss, it’s deliberate choice
Post reply on HN