Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Our position on open-weights models

anthropic.com

Post reply on HN
Print view View original

341–350 of 1001 posts

Re: Our position on open-weights models

#341

Post by verdverm »

ajyoon wrote: post #172

To everyone here pushing for total proliferation of open models -- what should be done about open weight bioweapon and cyber-offense capabilities? Is it simply the cost of freedom that we should allow attackers to access these tools? The OpenAI / Hugging Face incident shows what a GPT 5.6 level model can do off the leash; within ~6 months, open weight models will match this and every bad actor under the sun will be a…

> what should be done about open weight bioweapon

The same thing we do about bomb making today, certain ingredients are restricted and/or monitored. Bioengineering is a bigger lift to operationalize.

In other words, don't ban knowledge, make certain applications or ingredients illegal or highly regulated.

Re: Our position on open-weights models

#342

Post by K0balt »

Sure, if you’re going to sell an open-weight model over API in the USA it should refuse certain things.

Defensive cybersecurity should not be one of them, in fact, it should be required to provide defensive cybersecurity assistance on demand. Anthropic and OpenAI both fail miserably at assisting US companies to protect themselves from cyberattack.

As far as what I run on my own, not for sale over API, stay off of my lawn.

Re: Our position on open-weights models

#343

Post by qweqwe14 »

ajyoon wrote: post #172

To everyone here pushing for total proliferation of open models -- what should be done about open weight bioweapon and cyber-offense capabilities? Is it simply the cost of freedom that we should allow attackers to access these tools? The OpenAI / Hugging Face incident shows what a GPT 5.6 level model can do off the leash; within ~6 months, open weight models will match this and every bad actor under the sun will be a…

[dead]

Re: Our position on open-weights models

#344

Post by machinist5 »

areoform wrote: post #186
cogman10 wrote: post #28

> Anthropic has never advocated for a ban on open-weights models. > All sufficiently capable models, open and closed, should go through mandatory safety testing. Yeah, this is anthropic advocating for a ban on open weight models. Who runs this test? What happens if this test is too costly or the administrator refuses to allow certain people to participate. This is exactly how the US has banned goods in the past, by r…

When Fable was yanked, it was said to be (in part) due to the "jailbreak" of instructing Fable to "fix this code" — https://news.ycombinator.com/item?id=48552687 Dumb question. If "Mythos-class" models are such a problem, then... why not just let it fix everyone's code? There can't be more than a few million to tens of millions software businesses / services / regularly used F/OSS projects on Earth. Why not just give…

> Most people aren't malicious. If you, as an engineer, consultant, founder, business owner, or maintainer, were given access to Mythos' capabilities wouldn't you ask it to fix your code?

1. Some do not want to use LLMs because of grave ethical concerns.

2. Some do not want to use LLMs because of copyright concerns. Google v Oracle looms large in the background.

3. You presume the outcome of Fable / Mythos is a net positive for a FOSS project. Reviewing a firehose of code written without the context of the values and considerations of a particular project shaped over years or sometimes decades of formal and informal decisions is not necessarily the best use of the maintainers time.

Re: Our position on open-weights models

#346

Post by aprentic »

I'm curious how he would propose implementing this.

The US could ban connections to foreign AI providers and force US providers to submit to audits. Presumably, Chinese providers would see a rise in VPN traffic.

People can build fairly hefty home inference machines for the price of a small car and those will get better and cheaper. Are they going to try to stop people from downloading the weight files?

Re: Our position on open-weights models

#347

Post by Cookingboy »

soundworlds wrote: post #257

What Dario misses time and time again, is that people don't trust the US to create aligned AI anymore. His entire strategy rests on the assumption that the US (and their government) are exceptional. This is clearly false to the rest of the world.

I mean in the Bloomberg interview he has made it very explicit that he fully believes in American Exceptionalism. He literally said AI being involved in bombing school girls in other countries are ok because he trusts the American military leadership.

According to him the safety and morality rule of the whole world should be written by America alone.

Which is why in the same interview he said he supports the U.S. foreign policy while calling China "an aggressive and war mongering regime".

>This is clearly false to the rest of the world.

It's clearly false to more and more Americans too. But since the oligarch class benefits first and foremost from U.S. government policies the propaganda will continue to go on.

Re: Our position on open-weights models

#348

Post by vcryan »

I guess they have to make some statement about this, but we don't have to care. This is like the zoo making a statement about the employment of clowns. Clowns a a circus thing, nobody asked the zoo's opinion.

Re: Our position on open-weights models

#349

Post by paxys »

badatnames wrote: post #124

I can't remember the last time (if ever) a company managed to go from golden goose to.. whatever this is.. so quickly. The permanent defensiveness in his presentation is really hard to swallow, it actively puts me off wanting to believe in or rely on their product line with Dario at the helm. I don't even understand the logic leading up to this post. Who was it even hoping to convince. Is it possible Anthropic is due…

Anthropic has always been like this. People just responded to the message better when it came from the quirky underdog rather than the trillion dollar behemoth.

Re: Our position on open-weights models

#350

Post by dnw »

sanxiyn wrote: post #269
modeless wrote: post #26

> All sufficiently capable models, open and closed, should go through mandatory safety testing What happens if a model fails the test? Surely one can use Kimi K3 for evil, somehow or other. What now? "Mandatory safety testing" implies consequences for failing, yet Dario has nothing to say about what the consequences should be. He says he doesn't advocate a ban but it's hard to imagine what his alternative would be if…

If a model fails the test, it should be banned. He is not advocating a ban of open-weight models. He is advocating a ban of models that fail mandatory safety testing. Seems reasonable and straightforward.

He is not advocating for banning models per se but the proposal makes a business model (i.e. serving open weight models) that is starting to work more expensive.
Post reply on HN