[deleted]
Our position on open-weights models
121–130 of 1001 posts
Re: Our position on open-weights models
#122> We should crack down on industrial-scale distillation operations. Also Anthropic: AI firm Anthropic agrees to pay authors $1.5bn to settle piracy lawsuit https://www.bbc.com/news/articles/c5y4jpg922qo
I generally disagree with the claim that distilling a model is equivalent to training on freely available internet content – mostly due to investment required to turn it into a model – but piracy is another story. Pretty inexcusable.
Re: Our position on open-weights models
#123So the argument is basically: This technology is too dangerous so only _we_ should have access to it. We’re the good guys and only we can ensure a safe use of this technology. Quis custodiet ipsos custodes?
> To summarize my and Anthropic’s position, we have not and are not advocating for a ban on open-weights models as a category. We should instead focus on keeping powerful chips out of authoritarian hands, stopping industrial-scale distillation, and requiring safety testing of all sufficiently capable models, open and closed.
Maybe I should’ve expanded a bit on my comment. I didn’t mean “we” as in Anthropic directly—even though they certainly are advocating for restrictions on how to train AI systems by proposing mandatory safety training—I mean the argument that American AI labs are somehow more responsible than their Chinese counterparts.
It’s especially jarring when just last week OpenAI—an American company—accidentally hacked Hugginface when performing safety testing on an upcoming model [1]. If they have the ability to turn off all guardrails when testing out their models—or when selling them to the military—then the safety training is only there for show. If they can pick and choose who should have access to their most powerful model, surely they are trying to act as the world police?
[1] https://openai.com/index/hugging-face-model-evaluation-secur...
Re: Our position on open-weights models
#124I can't remember the last time (if ever) a company managed to go from golden goose to.. whatever this is.. so quickly. The permanent defensiveness in his presentation is really hard to swallow, it actively puts me off wanting to believe in or rely on their product line with Dario at the helm. I don't even understand the logic leading up to this post. Who was it even hoping to convince. Is it possible Anthropic is due an oil change?
Re: Our position on open-weights models
#125> Anthropic has never advocated for a ban on open-weights models. > All sufficiently capable models, open and closed, should go through mandatory safety testing. Yeah, this is anthropic advocating for a ban on open weight models. Who runs this test? What happens if this test is too costly or the administrator refuses to allow certain people to participate. This is exactly how the US has banned goods in the past, by r…
There should be safety testing, but no guardrails that limit models for cyber or bio research.
Guardrails are not a safety measure, they are a pay-to-play scheme that allows the people with deep pockets to have access to offensive and defensive capabilities first.
Re: Our position on open-weights models
#126> We should crack down on industrial-scale distillation operations. Also Anthropic: AI firm Anthropic agrees to pay authors $1.5bn to settle piracy lawsuit https://www.bbc.com/news/articles/c5y4jpg922qo
I generally disagree with the claim that distilling a model is equivalent to training on freely available internet content – mostly due to investment required to turn it into a model – but piracy is another story. Pretty inexcusable.
If they paid for tokens say via subscriptions. Wouldn't that just be same as acquiring books and using them as "fair use" to train? I really see no difference.
Re: Our position on open-weights models
#127Anthropic will continue being a victim of their own naive positions on AI safety. They keep dancing around it but their communication is essentially pro-regulation if you read between the lines.
Or if you read the lines. "All sufficiently capable models, open and closed, should go through mandatory safety testing."
Re: Our position on open-weights models
#128“By virtue of being the good people everything we do is good, and if it happens to be in our best personal and financial interest then that is good too because it enables us to do more good. And if it’s to the detriment to others then it’s because you don’t understand the good behind it. Which is fine, because we’re good and you can trust that what we do is good because what we do is good by virtue of us being the good ones.”
Re: Our position on open-weights models
#129> Anthropic has never advocated for a ban on open-weights models. > All sufficiently capable models, open and closed, should go through mandatory safety testing. Yeah, this is anthropic advocating for a ban on open weight models. Who runs this test? What happens if this test is too costly or the administrator refuses to allow certain people to participate. This is exactly how the US has banned goods in the past, by r…
The entire safety evals industry is essentially funded and controlled by OpenAI/Anthropic. Notice that on recent models, they exclusively use internal testing or black box external vendors (e.g., Gray Swan) whose entire business is to serve OpenAI/Anthropic. And all these companies just share the same pool of researchers back and forth.
So, it's a cottage industry.
Re: Our position on open-weights models
#130[flagged]
Yeah, like bombing a children school, the CCP did that not long ago.