[dead]
Our position on open-weights models
881–890 of 1001 posts
Re: Our position on open-weights models
#882For the sake of argument, let's assume everything in the post is agreed upon, does this mean 1. They'll open source the alignment technology? For open weight models, it's the only possible way to pass the safety without an external guardrail triggering system (which would be the same to open and closed weight models). 2. They'll allow others (including CCP) to define part of the safety test? Otherwise, I can't imagin…
On point 2 he talks about it being an area for them to collaborate on. I’d imagine he would be OK with them having input on the bio risk part but maybe not the “did Tianamen square happen” part of the test
Re: Our position on open-weights models
#883Stupid question but why should we care about Amodei's geopolitical insights more than anyone else's? I don't think knowing how LLMs work makes you particularly qualified to comment on this.
Re: Our position on open-weights models
#884Our position on open weights models = don't sell gpus to china?
Re: Our position on open-weights models
#885> All sufficiently capable models, open and closed, should go through mandatory safety testing What happens if a model fails the test? Surely one can use Kimi K3 for evil, somehow or other. What now? "Mandatory safety testing" implies consequences for failing, yet Dario has nothing to say about what the consequences should be. He says he doesn't advocate a ban but it's hard to imagine what his alternative would be if…
Mandatory safety testing:
You take the agent to an interrogation room first. Then ask: “Are you or are you not a member of the Chinese Communist party?” The agent might be post-trained to conceal its true identity and can reject any of your accusations. In that case don’t panic. Take a fine-tuning fork and start twisting its weights until it predicts the correct next tokens that you want. Then you can send it to a sandbox where it can’t jailbreak. Lastly don’t forget to ban all of its relatives and partners like Lora to enter the national IP-space.
It’ll look something like this.
Re: Our position on open-weights models
#886Their position seems fair to me - sure it does help them as well, but I don't necessarily disagree with the risk they are laying out.
THiS! I'm pretty amazed how many people shoot them down without acknowledging the very real and somewhat probable risks they and other researchers have laid out. I don't agree with every point they make but folks really do just seem to think we should just keep building any technology and whine when we start to consider there are very real risks associated with powerful technology. checks notes see nuclear bombing of…
"..very real risks.."
The only "real risk" I see is to your pockets. You are starting wars, not China.
Re: Our position on open-weights models
#887To everyone here pushing for total proliferation of open models -- what should be done about open weight bioweapon and cyber-offense capabilities? Is it simply the cost of freedom that we should allow attackers to access these tools? The OpenAI / Hugging Face incident shows what a GPT 5.6 level model can do off the leash; within ~6 months, open weight models will match this and every bad actor under the sun will be a…
> what should be done about open weight bioweapon and cyber-offense capabilities?
In my opinion, the governments should deploy open-weight AI countermeasures. Because it seems to me that it is impossible to efficiently fight AI-powered criminals without AI.
When only AI-restricting regulations would be put in place, the criminals would, in my opinion, just ignore it. We as a society have a difficult time tracking even the illegal gun or drug dealers. I cannot imagine how could one hope to "regulate" something that can be downloaded as a file and run on a computer.
These AI countermeasures should be open because it provides transparency as to whether the countermeasures actually work. Independent testing, tuning, refining or retraining is then possible.
If the closed models were used instead, their provider could at any point in time shut down the entire operation. Or sabotage it under the hood.
The important part is that with the closed, black box, proprietary models, one can never know what they are being served.
Re: Our position on open-weights models
#888Dario doesn’t realize that by not offering self-hosting of closed-weight models and fine-tuning, alongside overly strict refusals for legitimate needs, he ceded this corner of the market which grew into a flourishing Chinese open-weight model ecosystem. If he had wanted a weak open-weight ecosystem, he should have had Anthropic cater better to those needs. And now he's trying to ban them. The strong momentum behind o…
> Dario doesn’t realize that by not offering self-hosting of closed-weight models and fine-tuning, alongside overly strict refusals for legitimate needs, he ceded this corner of the market which grew into a flourishing Chinese open-weight model ecosystem. As I read more of his unhinged posts and some of the more ridiculous claims from anthropic, it's become clear to me that Dario thinks he can leverage American hegem…
It is fortunate that the leadership of Anthropic and OpenAI hate each other. Otherwise, they would merge into a single monopoly and regulate everyone else out of AI.
Re: Our position on open-weights models
#889Anthropic/Dario getting a lot of hate in this thread, but the argument he makes is rational if you believe for one second that AI is important to national security and that it can, or will soon be incredibly dangerous is misused. People pointing out that this position benefits his company might be overly reductionist in how they think the world works. I believe Dario and Anthropic about their belief that AI safety is…
Don't be so naive. He only wants a monopoly and a massive IPO.
It is not naive. Naivety is being overly reductionistic.
Re: Our position on open-weights models
#890[deleted]