Live data from Hacker News

Our position on open-weights models

anthropic.com

71–80 of 1001 posts

Re: Our position on open-weights models

#71

"My primary concern is the risk that authoritarian governments—not solely the Chinese Communist Party (CCP), although the CCP is clearly the most capable threat—build AI models that are more powerful than those built by the US, and use them to achieve permanent military superiority or perpetrate incredibly deep repression of their own people." Hmmmm.

The US is already under an authoritarian regime, the only thing keeping the wheels on the bus is a very tired and barely-effective judiciary.

This is a temporary situation because either this regime is going to be knocked out of power, or it's going to follow through on its core Seven Mountains Mandate[1] theology and go full totalitarian.

Normally totalitarianism fears are overblown, but I think that these zealots would absolutely use the latest frontier models and pervasive surveillance to make The Handmaid's Tale look like a liberal fantasy by comparison.

[1] https://en.wikipedia.org/wiki/Seven_Mountain_Mandate

Re: Our position on open-weights models

#72

Their position seems fair to me - sure it does help them as well, but I don't necessarily disagree with the risk they are laying out.

It is consistent with their stated beliefs, unlike OpenAI who flip flop every 6 months on whether they support open source or not.

I think their biggest PR problem is that many people still think of loss-of-control/misalignment etc. as sci-fi. And the distillation arguments come off poorly because people feel as though all the labs have trained on their creative output without their consent, so they deserve to own the result in some way.

Re: Our position on open-weights models

#73
post #40

> We should crack down on industrial-scale distillation operations. Also Anthropic: AI firm Anthropic agrees to pay authors $1.5bn to settle piracy lawsuit https://www.bbc.com/news/articles/c5y4jpg922qo

but Dario is the good guy, distillation when done by Dario should be called innovation /s

Re: Our position on open-weights models

#74
I'm tired of being strung along on these silly narratives. I can't wait for open-weight models to be deployed around the world just so people like Dario will shut up about the mystical levels of power these models have.

Re: Our position on open-weights models

#78
He says that using the set of questions and answers from one model to train another model (deatilation) is cheaper than training the model without those datasets.

But he didn't mention that training any model from a set of texts and books is much cheaper than writing those books in the first place.

In other words, it's ok when Anthropic learns from others, but it is not ok when others learn from Anthropic.

Re: Our position on open-weights models

#79

“Anthropic has never advocated for a ban on open-weights models. Open-weights models that don’t have dangerous capabilities are a public good…” A bit confused on this part, what model doesn’t have dangerous capabilities?

It seems possible to deliberately not train on some offensive capabilities and still have a very useful model. For example, Opus 5 deliberately did not train on exploiting vulnerabilities, and so performed less well on exploits than Mythos, yet was equally proficient at finding such vulnerabilities, according to the Opus 5 system card in their "OSS-Fuzz" eval [1]. [1]: https://www.securityweek.com/anthropics-opus-5-n…

That's a good example, but I'm unsettled by Anthropic's growing refusals in the areas of chemistry and biology. If they think that scientific assistant models should be as unhelpful as Fable, because applied scientific knowledge is inherently dangerous, I don't want Anthropic or like-minded thinkers setting the standards for model safety.
Post reply on HN