Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Our position on open-weights models

anthropic.com

Post reply on HN
Print view View original

441–450 of 1001 posts

Re: Our position on open-weights models

#441

Post by jackdeansmith »

vitalyan8184 wrote: post #352
ajyoon wrote: post #172

To everyone here pushing for total proliferation of open models -- what should be done about open weight bioweapon and cyber-offense capabilities? Is it simply the cost of freedom that we should allow attackers to access these tools? The OpenAI / Hugging Face incident shows what a GPT 5.6 level model can do off the leash; within ~6 months, open weight models will match this and every bad actor under the sun will be a…

>To everyone here pushing for total proliferation of ... ...general-purpose computers ...unbreakable encryption ...unbackdoored communication ...unkillswitched vehicles ...unsurveiled dwellings >what should be done about ...? nothing >Do you seriously want this level of capabilities to be generally available with no guardrails? yes

What about uranium enrichment?

Re: Our position on open-weights models

#442

Post by jackdeansmith »

verdverm wrote: post #341
ajyoon wrote: post #172

To everyone here pushing for total proliferation of open models -- what should be done about open weight bioweapon and cyber-offense capabilities? Is it simply the cost of freedom that we should allow attackers to access these tools? The OpenAI / Hugging Face incident shows what a GPT 5.6 level model can do off the leash; within ~6 months, open weight models will match this and every bad actor under the sun will be a…

> what should be done about open weight bioweapon The same thing we do about bomb making today, certain ingredients are restricted and/or monitored. Bioengineering is a bigger lift to operationalize. In other words, don't ban knowledge, make certain applications or ingredients illegal or highly regulated.

And what are those ingredients for biology, which can be constrained as effectively as uranium enrichment? I'd argue there isn't anything which can easily be restricted or monitored.

Re: Our position on open-weights models

#443

Post by sfink »

1970-01-01 wrote: post #337
cogman10 wrote: post #28

> Anthropic has never advocated for a ban on open-weights models. > All sufficiently capable models, open and closed, should go through mandatory safety testing. Yeah, this is anthropic advocating for a ban on open weight models. Who runs this test? What happens if this test is too costly or the administrator refuses to allow certain people to participate. This is exactly how the US has banned goods in the past, by r…

Why wouldn't it be a scan, just as we have with all other open-source code? Why can't open-weight models be easily checked for evil alignment? Sophos, Symantec, Malwarebytes, etc. would surely leap at the chance to upsell you on their product.

Wait, you don't do that already? I don't overcomplicate it, I just run

    ai-grep -v "bad code"
on all of my source files and keep what's left. Why would you keep bad code around? If it breaks when I do that, I fix it, and try again until I achieve what I want with no bad code. Doesn't everyone do that?

Re: Our position on open-weights models

#444

Post by claaams »

cogman10 wrote: post #28

> Anthropic has never advocated for a ban on open-weights models. > All sufficiently capable models, open and closed, should go through mandatory safety testing. Yeah, this is anthropic advocating for a ban on open weight models. Who runs this test? What happens if this test is too costly or the administrator refuses to allow certain people to participate. This is exactly how the US has banned goods in the past, by r…

Won't this just incentivize companies to move operations outside of the US where these models aren't regulated?

Re: Our position on open-weights models

#445

Post by xboxnolifes »

areoform wrote: post #186
cogman10 wrote: post #28

> Anthropic has never advocated for a ban on open-weights models. > All sufficiently capable models, open and closed, should go through mandatory safety testing. Yeah, this is anthropic advocating for a ban on open weight models. Who runs this test? What happens if this test is too costly or the administrator refuses to allow certain people to participate. This is exactly how the US has banned goods in the past, by r…

When Fable was yanked, it was said to be (in part) due to the "jailbreak" of instructing Fable to "fix this code" — https://news.ycombinator.com/item?id=48552687 Dumb question. If "Mythos-class" models are such a problem, then... why not just let it fix everyone's code? There can't be more than a few million to tens of millions software businesses / services / regularly used F/OSS projects on Earth. Why not just give…

For starters, it's probably closer to $10,000 per codebase for Fable/Mythos for a full review. That would be around 5 years of their current spending I think.

They really want that level of spend coming into the company, not going out.

Re: Our position on open-weights models

#447

Post by btbuildem »

> the risk that authoritarian governments [...] build AI models that are more powerful than those built by the US, and use them to achieve permanent military superiority or perpetrate incredibly deep repression of their own people.

This is rich coming from a guy who signed deals with an authoritarian government that's in the midst of launching an unprecedented surveillance apparatus (hello flock, hi p4l4nt1r), having already deployed, nation-wide, an exorbitantly funded army of unaccountable shock troops under the guise of immigration enforcement.

The call is coming from inside the house, at 130dB, and your ears should be bleeding at this point.

Re: Our position on open-weights models

#450

Post by jackdeansmith »

pianopatrick wrote: post #116

I don't really see the link between use of AI and military superiority. My current understanding is a lot of current US military problems are due to rare earths supply chains. I don't see how AI would either help or hurt with that.

I see a very clear link, very capable AI models are clearly able to find and exploit cyber vulnerabilities at a speed and scale not possible before. In the right hands, that's clearly a weapon
Post reply on HN