Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Our position on open-weights models

anthropic.com

Post reply on HN
Print view View original

381–390 of 1001 posts

Re: Our position on open-weights models

#381

Post by az226 »

Dario doesn’t realize that by not offering self-hosting of closed-weight models and fine-tuning, alongside overly strict refusals for legitimate needs, he ceded this corner of the market which grew into a flourishing Chinese open-weight model ecosystem.

If he had wanted a weak open-weight ecosystem, he should have had Anthropic cater better to those needs. And now he's trying to ban them.

The strong momentum behind open-weight models from Chinese labs is now an unstoppable force. Instead of trying to ban it, Dario should consider a different approach: here are our cyber and bio alignment datasets and here are our RL recipes for making that alignment training work well. By openly sharing its data and code, Anthropic could help influence and shape these models before they are released, rather than treating the entire ecosystem as an enemy.

Cyber and bio alignment aren't Anthropic's competitive advantage, they are forms of risk management. There should therefore be little reason to keep this work private. If Anthropic genuinely believes these capabilities pose serious global risks, the more productive approach would be to welcome collaboration and help the broader ecosystem manage those risks better.

On refusals, the irony is that a company like Hugging Face had to use a Chinese open-weight model to fend off an illegal hacking of its platform (done by no other than OpenAI). If a company like Hugging Face can't get past the refusal gates, then everyone else doesn't stand a chance.

Re: Our position on open-weights models

#382

Post by parineum »

bryan0 wrote: post #362
cogman10 wrote: post #28

> Anthropic has never advocated for a ban on open-weights models. > All sufficiently capable models, open and closed, should go through mandatory safety testing. Yeah, this is anthropic advocating for a ban on open weight models. Who runs this test? What happens if this test is too costly or the administrator refuses to allow certain people to participate. This is exactly how the US has banned goods in the past, by r…

> Yeah, this is anthropic advocating for a ban on open weight models. This is an ungenerous take, and I think it's important to to recognize it's reasonable to support models that are both open and safe. How this would actually be achieved is unclear though. Dario is at least proposing a solution a solution, which is the model needs to pass safety testing. This is reasonable and I wouldn't conflate this with wanting…

> This is an ungenerous take

I think that's well earned.

Re: Our position on open-weights models

#383

Post by storus »

The speech he had at congress didn't sound very flattering towards the beginning of his today's statement:

https://www.youtube.com/watch?v=_i91NSOyxHM

He didn't mention outright banning open source LLMs, just that their safe release would be a much harder problem, which to me implied "the easiest way is to ban the open source models".

Re: Our position on open-weights models

#385

Post by sm-silversight »

bobjordan wrote: post #153

"In fact, the most dangerous model may be one that is trained in secret and handed only to the People’s Liberation Army for use in drones and the Ministry of State Security for surveillance and repression." I'm so sick of all this anti-China shilling. There's zero chance that whomever is in power in the U.S. won't use AI in drones and in FBI/CIA/local Police/etc., for surveillance and repression right here in the goo…

Flock shows that they're happy to do it and rub our noses in it.

Re: Our position on open-weights models

#386

Post by thinkingtoilet »

m3h wrote: post #207

Someone who until yesterday did not seem bothered by his technology being possibly used to bomb elementary girls school in another country seems to suddenly care about the repression of citizens in yet another country. No, we don't buy your virtue signaling. And we certainly don't need your better-than-thou opinions on this year's "nightmare scenarios".

I like how he cited the current Trump administration in section about stifling decent.

Re: Our position on open-weights models

#387

Post by bigyabai »

credit_guy wrote: post #370

I know it's unpopular, or unfashionable, but I agree with this letter. LLMs are becoming so powerful that they are dangerous. We've seen last week with the OpenAI hacking (by mistake) Hugging Face debacle. It is absolutely ok to have open weight models at the level of GPT-OSS-100B. That one was released one year ago, and I think it's still a strong one. GLM 5.2 is a whole new level, but it appears to still be safe. M…

> We've seen last week with the OpenAI hacking (by mistake) Hugging Face debacle.

If the biggest danger of LLMs is that they can hack traditional systems, there is no significant threat to humanity posed by releasing them in open-weight form. Security doesn't become less of a problem by making hacking even more criminal. That's what's an unsafe mindset looks like.

Re: Our position on open-weights models

#388

Post by le-mark »

ajyoon wrote: post #172

To everyone here pushing for total proliferation of open models -- what should be done about open weight bioweapon and cyber-offense capabilities? Is it simply the cost of freedom that we should allow attackers to access these tools? The OpenAI / Hugging Face incident shows what a GPT 5.6 level model can do off the leash; within ~6 months, open weight models will match this and every bad actor under the sun will be a…

> Is it simply the cost of freedom that we should allow attackers to access these tools?

Bad actors WILL have access. The question is will these mega corps stop innovation?

Re: Our position on open-weights models

#389

Post by TheArcane »

paxys wrote: post #81

Hate to break it to you Dario but the way things stand right now the world at large trusts the Chinese establishment a lot more than the American one.

I cheer every time there's a new Deepseek, Qwen, Kimi, Z.ai release. Intelligence shouldn't be owned and charged for by a few

If it were up-to these silicon valley tech bros, they'd find a way to meter and charge for the air we breathe.

Re: Our position on open-weights models

#390

Post by jmward01 »

A lot of people have a lot of concerns with AI, its capabilities and with how it is used now and what it will lead to in the future. For good reason. But the question is, do we have a strategy that is actually useful? If so, what is it? I think nature shows us some answers in ecosystems.

Diverse ecosystems can absorb shocks. Diverse ecosystems are a sign of health of that ecosystem. When an invasive species comes into a healthy, diverse, ecosystem it doesn't mean that it isn't disrupted, but it does mean that it is far more likely to emerge with a lot of its diversity intact. In fact, it is likely to emerge even stronger because it can absorb that new shock and incorporate it, adding to its diversity. The balance may be changed, but the ecosystem survives or even thrives.

Nature also likes to show us that artificial barriers rarely last. You want to control a river? Good luck. It take constant maintenance to hold that flow in place and even then you are likely to get extremes that are made worse by your efforts because, eventually, somewhere in the system fails in a way you didn't anticipate. Then the water comes rushing in. Artificial barriers often have a way of building up tension over time, not reducing it, so that when a failure eventually happens it can be catastrophic. In other words, you had better really understand the system you are trying to control or else you can make things actively worse.

Relating this to the world now means, I think, that our best chance to minimize long term shock and maximize the chance that the diversity we have around us survives is to try to grow as healthy of an ecosystem as we can as quickly as possible. Lots of models large and small in lots of different hands is, I think, a better solution than artificial barriers restricting the variety and diversity of models and users. I think this is closer to an ecosystem solution and has a shot at working. Basically, I highly doubt we understand this situation enough to do a good job of controlling it with artificial barriers. Instead I think we are more likely to build catastrophic imbalances than we are to create the healthy ecosystem we really need.

Post reply on HN