191–200 of 1001 posts
“Questions like this should be answered empirically through rigorous pre-release testing, not assumed in advance.”
Exactly.
A bit off-topic from the core of the post, but: > At Anthropic we’re committed to cracking down on industrial-scale distillation through our own practices, including identifying and banning accounts that use our models in this way. This is challenging—for instance, the relevant accounts can often only be identified after substantial distillation has occurred, and distillation often involves creating large numbers of…
so Anthropic's ask is for US gov to ban open weight models so that its growth (and IPO) is not affected
> Anthropic has never advocated for a ban on open-weights models. > All sufficiently capable models, open and closed, should go through mandatory safety testing. Yeah, this is anthropic advocating for a ban on open weight models. Who runs this test? What happens if this test is too costly or the administrator refuses to allow certain people to participate. This is exactly how the US has banned goods in the past, by r…
There are many other regulated industries, like drugs (the FDA), cars (NHTSA and EPA), airplanes and rocket launches (the FAA), radios (the FCC) and so on. That's not unusual. Regulation is normal for stuff that might be dangerous.
The reality is much less confusing: Anthropic CEO does not wish for models with similar (or greater) capabilities compared to his own closed and overpriced ones to be widely released. Simply because that will affect Anthropic's bottom-line.
Anthropic and all other "model" companies have nothing making them special beyond privileged access to chips so obviously they want to restrict what models are out there and more importantly who can produce new ones. Without these restrictions, it's only a matter of time before the multi-hundred billions valuations simply evaporate while they are still holding the bag.
It seems obvious to me that the whole question of regulating a file is a bit silly. Any law that pushes against these things will just make it more secretive. I'm not sure that's any better.
> Anthropic has never advocated for a ban on open-weights models. > All sufficiently capable models, open and closed, should go through mandatory safety testing. Yeah, this is anthropic advocating for a ban on open weight models. Who runs this test? What happens if this test is too costly or the administrator refuses to allow certain people to participate. This is exactly how the US has banned goods in the past, by r…
I mean you're assuming this is even possible. I don't really care what the US admin does. If someone releases a powerful open source model I'll run it. Good luck trying to stop everyone doing that.
Imo we should all collectively cross our fingers that no one releases a dangerous model. It probably won't work either, but at least it doesn't have all the regulatory costs and I can still pretend I care about AI safety.
Every single risk he identifies as a concern regarding China is exactly my concerns with the US having absolute control. Literally the exact same concerns
> Anthropic has never advocated for a ban on open-weights models. > All sufficiently capable models, open and closed, should go through mandatory safety testing. Yeah, this is anthropic advocating for a ban on open weight models. Who runs this test? What happens if this test is too costly or the administrator refuses to allow certain people to participate. This is exactly how the US has banned goods in the past, by r…
The entire safety evals industry is essentially funded and controlled by OpenAI/Anthropic. Notice that on recent models, they exclusively use internal testing or black box external vendors (e.g., Gray Swan) whose entire business is to serve OpenAI/Anthropic. And all these companies just share the same pool of researchers back and forth.
I expect some of those tests (prolly not public) will basically be "wokeness" tests or "PC correctness" tests or "western media filter" tests.
China has different objectives. Sure.
I'm not sure one is safer than the other; I would know which one to go to if I want to research on topic that are viewed very different on both sides of this "new iron curtain".