Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Our position on open-weights models

anthropic.com

Post reply on HN
Print view View original

311–320 of 1001 posts

Re: Our position on open-weights models

#311

Post by usef- »

areoform wrote: post #186
cogman10 wrote: post #28

> Anthropic has never advocated for a ban on open-weights models. > All sufficiently capable models, open and closed, should go through mandatory safety testing. Yeah, this is anthropic advocating for a ban on open weight models. Who runs this test? What happens if this test is too costly or the administrator refuses to allow certain people to participate. This is exactly how the US has banned goods in the past, by r…

When Fable was yanked, it was said to be (in part) due to the "jailbreak" of instructing Fable to "fix this code" — https://news.ycombinator.com/item?id=48552687 Dumb question. If "Mythos-class" models are such a problem, then... why not just let it fix everyone's code? There can't be more than a few million to tens of millions software businesses / services / regularly used F/OSS projects on Earth. Why not just give…

They are doing that (see their project glasswing over the past few months), but there's a lot more code in the world than you realise.

The problem with rolling it out is that bad and good actors can both use it at the same time, and bad actors will typically move faster than typical day-to-day software projects and patching schedules, so they set up glasswing to give access to the major producers and projects to patch their own software before it becomes available more widely (they've submitted tremendous numbers of security issues to open source projects)

Re: Our position on open-weights models

#314

Post by exabrial »

I disagree with 100% of everything said in this article.

> My primary concern is the risk that authoritarian governments—not solely the Chinese Communist Party (CCP)....

This is why open weights win. See Linux and how it's taken over the world. Your business model will need to change eventually. Instead you're advocating trying to exterminate competition via regulation and fear mongering.

> My secondary concern is the risk that powerful AI models may be misused to carry out cyberattacks or biological attacks

Yawn... this is getting old.

> We should not sell powerful chips or chipmaking equipment to China

For as someone as smart as you guys, you sure lack common sense. China is just going to develop these technologies organically then and you lose 100% of control. It's already happened in reverse with things like Solar, rare earth minerals, etc. China flooded our market, destroyed our ability to produce things, now holds the keys. One thing they DIDNT do was stop trading to the US. They killed us with cheap goods.

> We should crack down on industrial-scale distillation operations.

Thats your problem, not my problem. Also, irony meter here hitting 11 about all those pirated books you stole...

> All sufficiently capable models, open and closed, should go through mandatory safety testing

Oh, fuck, no. This is a crackdown on free speech and rights of people to do whatever they want. My right to free speech means I'm allowed to write whatever computer program I want, no matter what its size is or how "sufficiently advanced" it is. Individual rights always win.

I really hope people don't believe this garbage. For a company with a great product, this is absolute nonsense.

Re: Our position on open-weights models

#315

Post by erwald »

GodelNumbering wrote: post #155

In the first paragraph, > Anyone who has read my past writing should know that I don’t regard such bans as a useful measure, Later (on banning chip sales to china) > we should crack down on the rampant smuggling and workarounds used to obtain access to such chips. If you truly believe that bans don't work, the same applies to hardware too. Furthermore, Dario says later "To address these concerns, I do support the fol…

Bans on Chinese open weight models being used in the US and bans on AI chips and semiconductor manufacturing equipment being exported to China are two extremely different things, and it's not inconsistent in any way to oppose one and endorse another.

Re: Our position on open-weights models

#316

Post by MrCheeze »

mjorgers wrote: post #16

So the argument is basically: This technology is too dangerous so only _we_ should have access to it. We’re the good guys and only we can ensure a safe use of this technology. Quis custodiet ipsos custodes?

That is in fact Anthropic's entire reason for existence, the belief that AGI is too dangerous to be controlled by OpenAI/Sam Altman. It naturally follows that it would also be too dangerous to be in the hands of literally everyone on earth.

Re: Our position on open-weights models

#317

Post by ethin »

cogman10 wrote: post #28

> Anthropic has never advocated for a ban on open-weights models. > All sufficiently capable models, open and closed, should go through mandatory safety testing. Yeah, this is anthropic advocating for a ban on open weight models. Who runs this test? What happens if this test is too costly or the administrator refuses to allow certain people to participate. This is exactly how the US has banned goods in the past, by r…

Not to mention: what are the "safety" standards we should enforce? And how should those standards even be enforced?

Re: Our position on open-weights models

#318

Post by MiSeRyDeee »

Pretty laughable. Dario seems to be missing one fundamental point with all this gibberish - if CCP is so bad why would they release the model open-weights for everyone to examine? Actually, the letter would make much more sense if what's actually happening is reversed, i.e. they are releasing open-weights model for the public good and China is distilling their model for its evil purpose.

Re: Our position on open-weights models

#319

Post by dnw »

> Open-weights models—it does not matter whether they come from China or anywhere else—do potentially present a higher risk than closed models

I don’t think this is open or closed; this is aligned and unaligned. I bet Grok would be as open as any open weight models to answering questions.

Re: Our position on open-weights models

#320

Post by pylua »

At the end of the day it hurts U.S. consumers to not have Chinese cars mainstream in the market. We lose out on features and stagnate on innovation because we are not pushed to compete.

I can’t imagine this would be any different — banning open weight models would hurt us in the long run. The point is to beat the competition, not suppress it.

There should be no limits on open or custom models. Too often safety is a synonym for surveillance and control. It’s a natural consequence, intended or not.

I am curious… why can’t distillation be stopped?

As a side not Im not against protectionism, but it has to be across the board and the same in all industries with no excrptions. We’ve let all these industries die on the vine due to cheap cost in foreign countries. It could very well happen to ai.

Post reply on HN