Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Our position on open-weights models

anthropic.com

Post reply on HN
Print view View original

511–520 of 1001 posts

Re: Our position on open-weights models

#511

Post by marcus_holmes »

pylua wrote: post #380
ajyoon wrote: post #172

To everyone here pushing for total proliferation of open models -- what should be done about open weight bioweapon and cyber-offense capabilities? Is it simply the cost of freedom that we should allow attackers to access these tools? The OpenAI / Hugging Face incident shows what a GPT 5.6 level model can do off the leash; within ~6 months, open weight models will match this and every bad actor under the sun will be a…

The software industry should be ashamed by the number of exploits that ai can find in software. It’s really an embarrassment. The software has to be built better.

Every single software engineer in the industry agrees with you.

Every single project manager disagrees.

Don't blame the engineers, we were specifically instructed and paid to build things fast and cheap, and every time we argued for good we were shouted down.

Re: Our position on open-weights models

#512

Post by sfink »

lukewarm707 wrote: post #312

dario, most of the world wants CHINA to win because the USA is the bad guy. you should be worried about the USA having these models.

Anthropic is a US company, so it is lobbying the US to enact regulations to serve both its interests and the interest of safety. What else should it be doing? I imagine Anthropic (or at least many of the people who work there) are fully aware that USA is the bad guy right now, but that doesn't change what levers they have and do not have available.

Re: Our position on open-weights models

#513

Post by theptip »

GodelNumbering wrote: post #155

In the first paragraph, > Anyone who has read my past writing should know that I don’t regard such bans as a useful measure, Later (on banning chip sales to china) > we should crack down on the rampant smuggling and workarounds used to obtain access to such chips. If you truly believe that bans don't work, the same applies to hardware too. Furthermore, Dario says later "To address these concerns, I do support the fol…

> If you truly believe that bans don't work, the same applies to hardware too.

No, software and hardware are different. You can very plausibly prevent smuggling of physical objects, and you very obviously cannot prevent smuggling/diffusion of open source bits-on-disk.

Re: Our position on open-weights models

#514

Post by sreekanth850 »

This is nothing but a post made by the scared CEO of a company that is now facing fierce competition from Chinese labs and losing its competitive moat, nothing more, nothing less.

If decades of fighting have failed to stop piracy, I’m sure nobody can stop China from sourcing high end chips. Unlike piracy, I’m happy that Chinese labs are releasing open-source models, so people in developing countries are no longer at the mercy of this capitalist bullshit.

Re: Our position on open-weights models

#515

Post by cogman10 »

tyre wrote: post #374
cogman10 wrote: post #28

> Anthropic has never advocated for a ban on open-weights models. > All sufficiently capable models, open and closed, should go through mandatory safety testing. Yeah, this is anthropic advocating for a ban on open weight models. Who runs this test? What happens if this test is too costly or the administrator refuses to allow certain people to participate. This is exactly how the US has banned goods in the past, by r…

What are you suggesting as an alternative? Everyone seems to want some fairytale world where there are open models, they’re all safe according to that person’s exact balance of risk and capabilities, and no one except the author or cynics are acting in good faith. What Dario lays out is very reasonable _of course_ the devil is in the details, but between him and Altman, there’s a clear divide on who to trust.

> What are you suggesting as an alternative?

This isn't something that can be regulated. Plain and simple.

If a dangerous model can exist and is being developed by a foreign adversary then no level of US law will stop said model from making it's way to hardware capable of running it. Even if direct transmission is impossible, it's FAR too easy to shove a model's data onto 1 or more thumb drives or hard drives and smuggle them pretty much anywhere in the world.

The only way to actually mitigate this sort of risk would be a global government with deep enforcement powers. That doesn't exist and won't exist. The UN is the closest we have to anything like that and... yeah...

Dario is fear mongering. He knows his proposals won't be even a minor speed bump in a dangerous model being created and used. His "reasonable" proposals are for the US market only and are literally just to create a bigger moat for his own company. They don't make anyone safer other than his shareholder's wallets. The only people he stops these dangerous models from being used by are people that won't be using them in a dangerous fashion.

Re: Our position on open-weights models

#516

Post by paxys »

manoDev wrote: post #336
ajyoon wrote: post #172

To everyone here pushing for total proliferation of open models -- what should be done about open weight bioweapon and cyber-offense capabilities? Is it simply the cost of freedom that we should allow attackers to access these tools? The OpenAI / Hugging Face incident shows what a GPT 5.6 level model can do off the leash; within ~6 months, open weight models will match this and every bad actor under the sun will be a…

This Pandora box is already open. Any argument about guardrails now are only attempts to create an artificial monopoly or keep this power in the hand of a single nation state, and _that_ is the absolute worst, most authoritarian future possible.

How is it already open? There has been ONE successful AI-driven cyberattack, and that was done by a model in testing that no one has access to. What would the picture be today if OpenAI and Anthropic had released 5.6 Sol and Mythos to everyone with no cyber restrictions (which is what everyone here was advocating for)?

Re: Our position on open-weights models

#517

Post by gr_norm »

adastra22 wrote: post #420
ajyoon wrote: post #172

To everyone here pushing for total proliferation of open models -- what should be done about open weight bioweapon and cyber-offense capabilities? Is it simply the cost of freedom that we should allow attackers to access these tools? The OpenAI / Hugging Face incident shows what a GPT 5.6 level model can do off the leash; within ~6 months, open weight models will match this and every bad actor under the sun will be a…

The bioweapon thing is absolute movie plot fiction. Go speak to some biologists about this and they'll set you straight. Cyber capabilities go both ways. Better offensive capabilities means better penetration testing by white hat security experts, which leads to better protections.

The effective altruism/rationalism/AI xrisk people have always had a shockingly poor grasp on subjects outside computer science, despite their attempts to speak on them. I don't blame the actual biologists and chemists working at the frontier labs for wanting to skim a few bucks off all the money flying around, though! I know a couple who've had not-so-kind words to say about their employers' intelligence.

I suspect there's at least some "telling the bosses what they want to hear" going on. A massive financial incentive exists to exaggerate and fearmonger even internally to the company, because it makes you and your job seem more important.

Re: Our position on open-weights models

#518

Post by cogman10 »

wolvoleo wrote: post #424
cogman10 wrote: post #28

> Anthropic has never advocated for a ban on open-weights models. > All sufficiently capable models, open and closed, should go through mandatory safety testing. Yeah, this is anthropic advocating for a ban on open weight models. Who runs this test? What happens if this test is too costly or the administrator refuses to allow certain people to participate. This is exactly how the US has banned goods in the past, by r…

Besides, Banning those models in the US does nothing to protect from other actors using them. That doesn't help in any way. It also doesn't stop non law abiding US citizens from having access to them. So basically it just stops the 'good guys' not the bad guys. I say good guys from a US perspective of course.

I'd argue it won't even stop people in the US from using those models. Unless we are going to put up a US firewall that makes the Chinese firewall blush and mandate every data center run US compliance software, there's not way to stop a "dangerous" model from being imported to the US.

This only stops the likes of OpenRouter from selling access to models. That's it. I'm sure an EU or chinese based alternative will pop up overnight (if they don't already exist).

Re: Our position on open-weights models

#519

Post by langs »

modeless wrote: post #26

> All sufficiently capable models, open and closed, should go through mandatory safety testing What happens if a model fails the test? Surely one can use Kimi K3 for evil, somehow or other. What now? "Mandatory safety testing" implies consequences for failing, yet Dario has nothing to say about what the consequences should be. He says he doesn't advocate a ban but it's hard to imagine what his alternative would be if…

"mandatory safety testing" is an impractical ideal, it's not workable in real world. Like any technology, LLMs are dual-use tools capable of both beneficial and malicious applications—a fundamental reality that human intent cannot change.
Post reply on HN