Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Our position on open-weights models

anthropic.com

Post reply on HN
Print view View original

541–550 of 1001 posts

Re: Our position on open-weights models

#541

Post by crossroadsguy »

cogman10 wrote: post #28

> Anthropic has never advocated for a ban on open-weights models. > All sufficiently capable models, open and closed, should go through mandatory safety testing. Yeah, this is anthropic advocating for a ban on open weight models. Who runs this test? What happens if this test is too costly or the administrator refuses to allow certain people to participate. This is exactly how the US has banned goods in the past, by r…

Thing is world has learned from the collective past experiences. Esp. with stuff like nuclear technology and nuclear weapons. I hope everyone here remembers/knows shit like CTBT. At least some countries were smart enough to not fall for that in the past knowing what it would mean if they didn't have it and it shows.

Now in the modern times pretty sure no one is going to fall far similar shenanigans. Even though some countries might sign some notional MoUs or some sort of CAIBT (Comprehensive AI Ban Treaty. Translation: "Only US and US companies get to develop and decide AI on Gaad's planet"), they/we already know that an agreement means squat only if you are weak enough to let someone enforce that on you.

Re: Our position on open-weights models

#542

Post by nullbio »

mjorgers wrote: post #16

So the argument is basically: This technology is too dangerous so only _we_ should have access to it. We’re the good guys and only we can ensure a safe use of this technology. Quis custodiet ipsos custodes?

Yeah, which is ironic considering their track record too. The most recent one is every shared Claude chat has been indexed by Google and is searchable on the search results. Then there was the RCE in their CLI that they had for a year and was silently patched without any sort of disclosure.

Re: Our position on open-weights models

#543

Post by diebillionaires »

This is just another case of a business insisting their people should be in charge of the evaluation of the safety. Worked really well for Boeing right?

Also I find it incredibly difficult to hear any company in the US worry about repression of people when financial repression is happening here. I’ve been to China and seen what the services to the public are like. Meanwhile the US funds and employs AI weapons in an ongoing genocide.

Re: Our position on open-weights models

#545

Post by potsandpans »

az226 wrote: post #381

Dario doesn’t realize that by not offering self-hosting of closed-weight models and fine-tuning, alongside overly strict refusals for legitimate needs, he ceded this corner of the market which grew into a flourishing Chinese open-weight model ecosystem. If he had wanted a weak open-weight ecosystem, he should have had Anthropic cater better to those needs. And now he's trying to ban them. The strong momentum behind o…

> Dario doesn’t realize that by not offering self-hosting of closed-weight models and fine-tuning, alongside overly strict refusals for legitimate needs, he ceded this corner of the market which grew into a flourishing Chinese open-weight model ecosystem.

As I read more of his unhinged posts and some of the more ridiculous claims from anthropic, it's become clear to me that Dario thinks he can leverage American hegemony to regulate his company into a monopoly.

He has an ethics vaguely influenced by effective altruism, and he appears to think that his ethical framework entitles him to make decisions on the behalf of humanity, for all of humanity.

Literally a bond villain.

Re: Our position on open-weights models

#547

Post by unscaled »

cogman10 wrote: post #28

> Anthropic has never advocated for a ban on open-weights models. > All sufficiently capable models, open and closed, should go through mandatory safety testing. Yeah, this is anthropic advocating for a ban on open weight models. Who runs this test? What happens if this test is too costly or the administrator refuses to allow certain people to participate. This is exactly how the US has banned goods in the past, by r…

For this testing to be really effective at stopping "dangerous and misaligned" models from leaking out, you need a mechanism for banning failed models that prevent them from being released in the first place, not just prevent US companies from using them.

The only way to stop this from happening is blocking the model's release at the first place. Which requires China agreeing to the same framework. Dario says exactly the same thing himself.

So if he's being truthful here, he's not advocating for the type of ban people are talking about (usage ban). This kind of ban would be helpful to Anthropic's business in the short term, but it won't prevent Chinese models from improving, and it won't prevent them from getting money selling to other countries.

He is openly advocating for an international effort to enforce tests on public models, but I think this is highly unlikely in the current climate. Even if both the US and China agree that public models should be prevented from being used in designing bioweapons, they need to agree on a test and enforcement framework and that requires a lot of negotiation and trust. I don't see this as likely in the near future.

Re: Our position on open-weights models

#549

Post by rileymat2 »

YmiYugy wrote: post #202
cogman10 wrote: post #28

> Anthropic has never advocated for a ban on open-weights models. > All sufficiently capable models, open and closed, should go through mandatory safety testing. Yeah, this is anthropic advocating for a ban on open weight models. Who runs this test? What happens if this test is too costly or the administrator refuses to allow certain people to participate. This is exactly how the US has banned goods in the past, by r…

Yeah, seems pretty likely. Anthropic will make the case that their models should be evaluated with the safety layer in front, because that is the only way the model is available whereas open weight models need to pass the same test just on the weights. The economic implications will be rather large, but in terms of security it seems inconsequential. The most compelling argument would be that by limiting the use of op…

> The US government can restrict access with IP blocks and limit inference capacity with export controls, but these measures are not effective in deterring malicious actors.

But aren't we talking about import controls, and the import of information itself? This has serious First Amendment ramifications.

Post reply on HN