Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Our position on open-weights models

anthropic.com

Post reply on HN
Print view View original

391–400 of 1001 posts

Re: Our position on open-weights models

#391

Post by elliotec »

rramach wrote: post #358

Wow, this comment thread clearly shows that at least Anthropic has not been a great communicator. If one reads this with a charitable lens, Dario is simply saying that 1) Nation state actors are a threat which needs to be combatted by chip bans and distillation prevention and 2) open-weight models can pose biological risk. One may or may not agree with item 1 but item 2 above should have broad support given the unkno…

Closed-weight models can also pose biological risk, arguably more so than open-weight models, given the fact that they're being used by state actors (the United States military) to kill people right now.

Who should we fear more? All of collective humanity with the keys to build destructive (and defensive) stuff with AI, or small groups of elites, billionaires, and state actors who have the monopoly on violence and want to control the keys?

Open-weight models collectivize access and ability to do more for a greater good, and the expense of a frankly low-risk possibility that some randos want to use it for very bad things.

Closed-weight models keep the control in the hands of the few that actually are doing the harm to the world, and the rest of us have no way to stop it or defend.

Re: Our position on open-weights models

#392

Post by georgemcbay »

pianopatrick wrote: post #116

I don't really see the link between use of AI and military superiority. My current understanding is a lot of current US military problems are due to rare earths supply chains. I don't see how AI would either help or hurt with that.

> My current understanding is a lot of current US military problems are due to rare earths supply chains.

Most current US military problems are due to the incompetence of its current civilian leadership.

Re: Our position on open-weights models

#393

Post by onlyrealcuzzo »

cogman10 wrote: post #28

> Anthropic has never advocated for a ban on open-weights models. > All sufficiently capable models, open and closed, should go through mandatory safety testing. Yeah, this is anthropic advocating for a ban on open weight models. Who runs this test? What happens if this test is too costly or the administrator refuses to allow certain people to participate. This is exactly how the US has banned goods in the past, by r…

The goal is to make the safety tests cost $100M+, so that no one can release a model legally useable for a large portion of the world, unless they charge high enough prices, to the point where no one would use it, thus no competition.

Re: Our position on open-weights models

#394

Post by nxtfari »

Surprisingly incoherent for Anthropic and Dario (cue peanut gallery — “always has been!” No, I don’t think so. I think this is new).

It seems to me like there is just no good answer to how one could possibly stop open weight models from being used for nefarious purposes. How are you going to enforce guardrails on open source? The only way is to turn the USA into a 1984-type totalitarian surveillance state (even more so than it is). Unable to say that, we just get this floundering instead. How long is not giving them chips going to slow them down? Until we RSI? Then what? Just because RSI runs off the exponential doesn’t mean that the eventual open-weight Moonshot Mythos won’t be able to make bioweapons. Genuinely what is the endgame.

Re: Our position on open-weights models

#395

Post by anduril22 »

YmiYugy wrote: post #202
cogman10 wrote: post #28

> Anthropic has never advocated for a ban on open-weights models. > All sufficiently capable models, open and closed, should go through mandatory safety testing. Yeah, this is anthropic advocating for a ban on open weight models. Who runs this test? What happens if this test is too costly or the administrator refuses to allow certain people to participate. This is exactly how the US has banned goods in the past, by r…

Yeah, seems pretty likely. Anthropic will make the case that their models should be evaluated with the safety layer in front, because that is the only way the model is available whereas open weight models need to pass the same test just on the weights. The economic implications will be rather large, but in terms of security it seems inconsequential. The most compelling argument would be that by limiting the use of op…

> but these measures are not effective in deterring malicious actors

Wanting to use open weight models in light of commercially imposed export controls doesn't make for "malicious actors"

Re: Our position on open-weights models

#397

Post by bigyabai »

fearnot wrote: post #191

Finally, some sense. This is the only argument I have seen that genuinely engages with the problem and approaches it with humility, rather than charging ahead on the basis of assumptions and without a shred of evidence. OpenAI should have been the one making it. “Questions like this should be answered empirically through rigorous pre-release testing, not assumed in advance.” Exactly.

> rather than charging ahead on the basis of assumptions and without a shred of evidence.

Anthropic's basis of assumption is the insinuation that LLMs can do things that we've never seen before, and that they can't tell us what it is. It sounds like you're also siding with an organization that has no evidence and relies on validating their own assumptions.

Re: Our position on open-weights models

#398

Post by htlemur_bobby »

Guys if we want safety we need to work with people, not make enemy of CCP. Geez this is extremely frustrating to see enemies being made. USA leads in torture and our prisons are worse than CCP prisons so USA is the worse issue. I recommend Anthropocene stop fundraising and do the right thing which is open source all.

Re: Our position on open-weights models

#399

Post by robviren »

YmiYugy wrote: post #202
cogman10 wrote: post #28

> Anthropic has never advocated for a ban on open-weights models. > All sufficiently capable models, open and closed, should go through mandatory safety testing. Yeah, this is anthropic advocating for a ban on open weight models. Who runs this test? What happens if this test is too costly or the administrator refuses to allow certain people to participate. This is exactly how the US has banned goods in the past, by r…

Yeah, seems pretty likely. Anthropic will make the case that their models should be evaluated with the safety layer in front, because that is the only way the model is available whereas open weight models need to pass the same test just on the weights. The economic implications will be rather large, but in terms of security it seems inconsequential. The most compelling argument would be that by limiting the use of op…

Regulatory capture and lobbies will keep you safe and you'll like it! The sudden surge is Washington dollars makes great sense with this context. Only way to keep the kids safe is attested compute all the way down. Don't you care for children???

Re: Our position on open-weights models

#400

Post by credit_guy »

bigyabai wrote: post #387
credit_guy wrote: post #370

I know it's unpopular, or unfashionable, but I agree with this letter. LLMs are becoming so powerful that they are dangerous. We've seen last week with the OpenAI hacking (by mistake) Hugging Face debacle. It is absolutely ok to have open weight models at the level of GPT-OSS-100B. That one was released one year ago, and I think it's still a strong one. GLM 5.2 is a whole new level, but it appears to still be safe. M…

> We've seen last week with the OpenAI hacking (by mistake) Hugging Face debacle. If the biggest danger of LLMs is that they can hack traditional systems, there is no significant threat to humanity posed by releasing them in open-weight form. Security doesn't become less of a problem by making hacking even more criminal. That's what's an unsafe mindset looks like.

Ok. What's your stance on gun ownership then? And by gun, I mean naval guns.
Post reply on HN