[deleted]
Our position on open-weights models
281–290 of 1001 posts
Re: Our position on open-weights models
#282Anthropic must be forced to open source all of the books they have scanned and burned.
For what? Would authors and publishers like that?
Re: Our position on open-weights models
#283As an American I’d gladly take payments from China to feed them my Claude transcripts for distillation. I’m surprised I haven’t heard of such an initiative.
Re: Our position on open-weights models
#284> > All sufficiently capable models, open and closed, should go through mandatory safety testing.
The problem with this is the cycles required to abliterate a model is significantly less than the cycles required to train a model.
This is the biggest reason why I'm against locking these models down / preventing their use. It's just delaying things by ~3-6mo, while in the process preventing legitimate use and adding red tape overhead.
Re: Our position on open-weights models
#285> We should crack down on industrial-scale distillation operations.
IP for me, not for thee.
Re: Our position on open-weights models
#286As an European I really dislike this consistent anti-Chinese narrative.
It's not China starting a war every few years, now causing a global economic fallout in Iran, it's not China threatening to annex Greenland/Canada/Panama, it's not China attacking foreign countries and kidnapping their leaders, it's not China who has been found to spy and intercept the communications and movements of its citizens and its allies and their leaders for the longest time, it's not China bombing civilians or stopping countries from obtaining basics like food, gas or oil.
I'm not saying that China is a paradise and US is bad, nor the contrary. We could make similar lists about most of the biggest countries out there.
I'm simply stating that this never ending US exceptionalism "US has to be the first and at the frontier of military, technology and this and that, but does not need to comply with the rules of the institutions it itself created" was already sickening and annoying before, but increasingly malign in the last decade and strongly accelerating as of recently.
I miss the time US CEOs were globalists and used their influence to advocate for a simpler world.
Re: Our position on open-weights models
#287> Anthropic has never advocated for a ban on open-weights models. > All sufficiently capable models, open and closed, should go through mandatory safety testing. Yeah, this is anthropic advocating for a ban on open weight models. Who runs this test? What happens if this test is too costly or the administrator refuses to allow certain people to participate. This is exactly how the US has banned goods in the past, by r…
This isn't actually about safety. This is just another example of pulling the ladder up so nobody else can follow
Re: Our position on open-weights models
#288Wow, so much cynicism and distrust in the comments, to the level of conspiracy theory, in my opinion. Yeah, this is the company that fairly recently refused to allow the government to use its models for autonomous weapons or mass surveillance, and refused to remove its guardrails.
Am not saying we should take what Dario is saying at face value, but he already has shown by his actual actions that he can be well intentioned. There might be elements of truth to what he’s saying.
Re: Our position on open-weights models
#289> Anthropic has never advocated for a ban on open-weights models. > All sufficiently capable models, open and closed, should go through mandatory safety testing. Yeah, this is anthropic advocating for a ban on open weight models. Who runs this test? What happens if this test is too costly or the administrator refuses to allow certain people to participate. This is exactly how the US has banned goods in the past, by r…
The entire safety evals industry is essentially funded and controlled by OpenAI/Anthropic. Notice that on recent models, they exclusively use internal testing or black box external vendors (e.g., Gray Swan) whose entire business is to serve OpenAI/Anthropic. And all these companies just share the same pool of researchers back and forth.
That doesn't sound like it describes SecureBio to me?
(Disclosure: I work at SecureBio, but not on the biological evals side.)
Re: Our position on open-weights models
#290It's absolutely reasonable to have safeguards on sufficiently dangerous models being released - if you disagree, can you explain your perspective?
I think it's wildly irresponsible to release models that are extremely capable at things like bio-weapons. Do you really think information anarchy is the answer?
The problem with open models compared to closed models is not about protecting profit - it's about protecting capability. Any open model can be retrained or fine-tuned for anything. There's no such thing as an open model that is both capable _and_ permanently safe when it comes to certain dangerous topics. It's not possible to prevent 'uncensoring' a model.