Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Our position on open-weights models

anthropic.com

Post reply on HN
Print view View original

571–580 of 1001 posts

Re: Our position on open-weights models

#571

Post by cogman10 »

jimbokun wrote: post #561
cogman10 wrote: post #28

> Anthropic has never advocated for a ban on open-weights models. > All sufficiently capable models, open and closed, should go through mandatory safety testing. Yeah, this is anthropic advocating for a ban on open weight models. Who runs this test? What happens if this test is too costly or the administrator refuses to allow certain people to participate. This is exactly how the US has banned goods in the past, by r…

It’s also how the FDA works. Ban new products until they have been proven safe. I think that also applies to AI products. It’s a hell if a lot better for the government to test and approve all models than having the industry “police itself” (lol)

The FDA regulates physical goods. They require literal factories and shipping to get these products anywhere. They can put stops on these products pretty easily. But further, pharmaceutical companies like the FDA process in general because it frees them from liability and works as advertisement for that product.

AI models are a finished product when the training is done. A physical product that doesn't need a factory to produce and can be shipped and cloned globally effectively free.

The better comparison is media. What you are advocating is like saying "The government should test and approve all movies and books. We shouldn't have those industries police themselves". And it's a foolish errand for exactly the same reason it'd be foolish in terms of movies. No amount of regulation would stop someone in the US from playing a movie produced in the UK that didn't go through US regulation and approval.

Re: Our position on open-weights models

#572

Post by killjoywashere »

For those not in the know on governance, there are international organizations formed around treaties for similar situations, the IAEA and OPCW come to mind. I'm not aware of similar constructs for bio, cyber, or AI.

On the plus side, for these newer threats, you've got more than 30 minutes before the end of civilization. On the down side, the energy levels for the launch events are much lower, so much harder to detect.

Re: Our position on open-weights models

#573

Post by paxys »

areoform wrote: post #186
cogman10 wrote: post #28

> Anthropic has never advocated for a ban on open-weights models. > All sufficiently capable models, open and closed, should go through mandatory safety testing. Yeah, this is anthropic advocating for a ban on open weight models. Who runs this test? What happens if this test is too costly or the administrator refuses to allow certain people to participate. This is exactly how the US has banned goods in the past, by r…

When Fable was yanked, it was said to be (in part) due to the "jailbreak" of instructing Fable to "fix this code" — https://news.ycombinator.com/item?id=48552687 Dumb question. If "Mythos-class" models are such a problem, then... why not just let it fix everyone's code? There can't be more than a few million to tens of millions software businesses / services / regularly used F/OSS projects on Earth. Why not just give…

$100 credit on Fable/Mythos will last a grand total of 10 minutes.

Re: Our position on open-weights models

#574

Post by bag_boy »

cogman10 wrote: post #28

> Anthropic has never advocated for a ban on open-weights models. > All sufficiently capable models, open and closed, should go through mandatory safety testing. Yeah, this is anthropic advocating for a ban on open weight models. Who runs this test? What happens if this test is too costly or the administrator refuses to allow certain people to participate. This is exactly how the US has banned goods in the past, by r…

He cited the Demis Hassabis’s framework for testing.

From Hassabis’s essay:

“It could establish a new Standards Body modelled on a federally overseen public-private partnership or self-regulatory organisation, much like the Financial Industry Regulatory Authority (FINRA), with a board that includes independent leading technical experts and open-source representatives.”

Re: Our position on open-weights models

#575

Post by parsimo2010 »

Such a cop-out. Dario, you got in the news because you were trying to say that Moonshot did something wrong by distilling Claude. You got in the news because you were trying to effectively make a "rules for thee but not for me" when you try to claim that you can train on whatever pirated works without any permission from the creators, but when someone uses your "work" to train without permission then all of a sudden it's a moral injustice. You can't have both.

Saying, "I'm not actually against open-weights, I'm against distillation" isn't addressing what made people mad. You're still trying to do some "rules for thee but not for me" nonsense and hiding behind some technicality. Trying to get the US government on your side to hold back your Chinese competition. If you had wanted the US government to support you, you should have let them make autonomous killer robots with Claude brains. They aren't going to help you, you didn't help them.

Just to be clear, I think that it is possible that literally everyone involved in this is full of crap and nobody is good. Dario and Anthropic are full of crap, for the reasons previously stated. The US government is full of lots of crap and should not be trying to make autonomous killer robots (not ever, but especially not when the bar for a "good" AI is knowing how many Rs are in strawberry or whether you should drive to a car wash). OpenAI is full of crap by signing some support for open weights models and they haven't touched open weights in a year (GPT-OSS released on Aug 5 so basically a year with no news). Google is less full of crap about the open weights stuff because of Gemma 4, but they are full of crap for a zillion other things I can't exactly feel good about them. So everyone sucks.

So cheers to Moonshot and Qwen and whoever else. Distill as much as you can and give us cheaper AI. I have the sneaking suspicion that a bunch of my tax money went to OpenAI and Anthropic in some shady way or another, and I want it back. I'll take it in the form of an open weights model being distilled from the fat cat models.

Re: Our position on open-weights models

#576

Post by antonvs »

jscott817 wrote: post #122
fuddle wrote: post #40

> We should crack down on industrial-scale distillation operations. Also Anthropic: AI firm Anthropic agrees to pay authors $1.5bn to settle piracy lawsuit https://www.bbc.com/news/articles/c5y4jpg922qo

I generally disagree with the claim that distilling a model is equivalent to training on freely available internet content – mostly due to investment required to turn it into a model – but piracy is another story. Pretty inexcusable.

> I generally disagree with the claim that distilling a model is equivalent to training on freely available internet content – mostly due to investment required to turn it into a model

The frontier labs all give free access to their models. Why does “investment” change anything? Anyone who’s ever produced any content, free or otherwise, has invested in doing so.

The only plausible issue I see is that if distillation is being done by creating many free accounts to work around limits on free accounts, that’s a bit… impolite? But if they really wanted to avoid that, they could eliminate free accounts, and require users to sign a real contract governing what they can do with the model.

Of course they don’t want to do that, so they’re stuck in the same world as the rest of us, and they don’t have any real basis to complain about it without being hypocritical.

Re: Our position on open-weights models

#577

Post by antonvs »

cmckn wrote: post #364
pianopatrick wrote: post #116

I don't really see the link between use of AI and military superiority. My current understanding is a lot of current US military problems are due to rare earths supply chains. I don't see how AI would either help or hurt with that.

This framing also builds a case for AI being a Second Amendment issue in the US, which seems to work against Anthropic here.

I’m going to start open carrying a DGX Spark.

Re: Our position on open-weights models

#578

Post by fractorial »

parsimo2010 wrote: post #575

Such a cop-out. Dario, you got in the news because you were trying to say that Moonshot did something wrong by distilling Claude. You got in the news because you were trying to effectively make a "rules for thee but not for me" when you try to claim that you can train on whatever pirated works without any permission from the creators, but when someone uses your "work" to train without permission then all of a sudden…

> Google is less full of crap about the open weights stuff because of Gemma 4, but they are full of crap for a zillion other things I can't exactly feel good about them. So everyone sucks.

Google catching this stray made me laugh, ha.

Post reply on HN