Actually this debate highlights how big the AI war is! my personal opinion is LLMs must be opensource and borderless, and ofcourse with given reputation i will use USA based Opensource model than the chinese one, but will they release opensource alternatives that is the question that will be answered in future
Our position on open-weights models
751–760 of 1001 posts
Re: Our position on open-weights models
#752The only ban that should be put in place is on models' general and specific capabilities. If they have open or closed weigths is irrelevant. With the current state of capabilities there is already plenty we can do (in terms of creation and destruction unfortunately) before we as a society need more powerful capabilities.
Once the limit is set, any non- governmental/military entity providing/hosting/using a more capabable model can be prosecuted.
Re: Our position on open-weights models
#753Amodei: Open model not so much bad, but open model in china hands very bad
Non-US world: But open model is in every ones hands.
Amodei: ... shut-up.
Re: Our position on open-weights models
#754Schrödinger's China at once is an evil entity looking to use AI for their own nefarious purposes yet also willing to cooperate with their main competitor to prevent other actors (who??) from achieving similar goals (all while under a chip embargo too!!) The reality is much less confusing: Anthropic CEO does not wish for models with similar (or greater) capabilities compared to his own closed and overpriced ones to be…
I don’t disagree with your point on Dario’s conflict of interest. I def think the models are expensive. But why call them overpriced? Compared to what? Even if we take the margin reports at face value, we don’t know their training costs, etc. Curious if this was more of an emotional take or if there’s actual evidence behind it.
> But why call them overpriced? Compared to what?
Compared to some available Chinese model I guess. For most common tasks the additional intelligence is marginal and the cost is around an order of magnitude higher.
Re: Our position on open-weights models
#755Schrödinger's China at once is an evil entity looking to use AI for their own nefarious purposes yet also willing to cooperate with their main competitor to prevent other actors (who??) from achieving similar goals (all while under a chip embargo too!!) The reality is much less confusing: Anthropic CEO does not wish for models with similar (or greater) capabilities compared to his own closed and overpriced ones to be…
I was reading the post and thinking "wow, that's pretty clear for a smart man used to writing for other smart men. it'll be really difficult to misunderstand". I read the first couple of comments and stand corrected.
A much simpler summary:
- open models good.
- smart models _can_ be bad
- smart open models that can do biotech work are dangerous. worth the hassle of certification _if_ we can get everybody on board with minimalist certification.
- banning open models just in US is neither good or bad: is stupid.
Re: Our position on open-weights models
#756Schrödinger's China at once is an evil entity looking to use AI for their own nefarious purposes yet also willing to cooperate with their main competitor to prevent other actors (who??) from achieving similar goals (all while under a chip embargo too!!) The reality is much less confusing: Anthropic CEO does not wish for models with similar (or greater) capabilities compared to his own closed and overpriced ones to be…
Yup, all that talk about "safety" seems to branch into 2 meanings
"Dangerous to our bottom line"
"We call it dangerous to hype up its abilities"
Re: Our position on open-weights models
#757I'm really wondering what nightmare scenario does he imagine with super intelligent drones.
I can already imagine it, a tiny drone carrying a 8x GPU rack thinking about life and deciding to go and build an idyllic society on a pacific island.
Re: Our position on open-weights models
#758In the first paragraph, > Anyone who has read my past writing should know that I don’t regard such bans as a useful measure, Later (on banning chip sales to china) > we should crack down on the rampant smuggling and workarounds used to obtain access to such chips. If you truly believe that bans don't work, the same applies to hardware too. Furthermore, Dario says later "To address these concerns, I do support the fol…
>all these moves commercially benefit Anthropic.
From a new "fast" company arising, you can sometimes see the way that before it started up there was nothing but "narrative" and that is what established the initial business model since there was nothing else yet. After some momentum is gained whether there is a pivot or not then the narrative going forward has to be aligned with the now more-well-proven business model.
From his leadership standpoint there are 3 big recommendations right now. That's what this message is all about.
>We should not sell powerful chips or chipmaking equipment to China
Well you and who else?
If there's not already somebody who is compromising the well-being of a nation in exchange for a handful of gold, with the ever-incresing glorification of greed & dishonesty at all costs it's only a matter of time for this one.
>We should crack down on industrial-scale distillation operations. Distillation is a much more compute-efficient process
Wait a minute, what's always been needed by everybody are more compute-efficient processes for everything. I've mentioned this before and it's been a while but back in 1980 it took less than a year to figure out I was going to need other chips that were not regular CPUs if I was going to get the most intelligent response from the silicon on a single square-foot of PCB. At the same time it was obvious you were never going to get far without what they now call "distillation", especially with only kilobytes of memory. Otherwise you would be wasting such stupidly large amounts of memory & storage there was no way you could really call it "intelligent". Now with ML & AI on the rise again there are so many people more deeply immersed than ever, and nothing has really contradicted these basic concepts yet, which have been easily recognizable since like forever.
>All sufficiently capable models, open and closed, should go through mandatory safety testing. The best way to address threat #2 is to just directly test models for cyber, biological, and alignment risks before release.
Righteous concept, and I'm always in favor of 100x the amount of testing in general normally done.
But "just" test says it pretty well, and those who are gifted enough to "draw the rest of the owl" freehand can test things the most skillfully until they are blue in the face. It's not going to help if an adversary decides not to test, or to enhance these exact things so it can have some kind of competitive advantage. Amodei does not ignore this and there is a footnote about it.
People realize that some of the elements that are coming to mind were expressed in some fairly early "science-fiction" so all this is nothing new
Still looking for the most intelligent responses I can get, since way before 1980 ;)
Re: Our position on open-weights models
#759> Anthropic has never advocated for a ban on open-weights models. > All sufficiently capable models, open and closed, should go through mandatory safety testing. Yeah, this is anthropic advocating for a ban on open weight models. Who runs this test? What happens if this test is too costly or the administrator refuses to allow certain people to participate. This is exactly how the US has banned goods in the past, by r…
Yeah, seems pretty likely. Anthropic will make the case that their models should be evaluated with the safety layer in front, because that is the only way the model is available whereas open weight models need to pass the same test just on the weights. The economic implications will be rather large, but in terms of security it seems inconsequential. The most compelling argument would be that by limiting the use of op…
> The most compelling argument would be [...] that it will reduce cases of accidents like the recent attack on Hugging Face.
So in the example provided: It was the closed model that did the attack, and they ended up using a self-hosted open model for their defense work. So the real world situation ended up exactly backwards from what you are inferring.
This was complicated by the fact that the protections in the closed frontier models meant that hugging face was denied their use in defense entirely.
This is called asymmetric capability, and it's probably the bigger threat.
Symmetric might be better: A rising tide lifts all ships, after all.
I'll grant that this is starting to look a lot like debates about (equal access to) guns, encryption, vaccination, genetics etc. The exact parameters determine the safest approach, and reasonable people may disagree.
Re: Our position on open-weights models
#760To everyone here pushing for total proliferation of open models -- what should be done about open weight bioweapon and cyber-offense capabilities? Is it simply the cost of freedom that we should allow attackers to access these tools? The OpenAI / Hugging Face incident shows what a GPT 5.6 level model can do off the leash; within ~6 months, open weight models will match this and every bad actor under the sun will be a…
There's no stopping bioweapons. Bioweapons are easy. The reason bioweapons aren't built is because very few biology nerds with sufficient lab skills are evil; and just having an LLM won't give you the lab skills to do it.
Anyone who can publish a gene technology/biomedicine paper can make a bioweapon. If you wrote a paper about how to make a bioweapon easily, it would be unpublishable not because of any danger, but because there wasn't enough novelty.