Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Our position on open-weights models

anthropic.com

Post reply on HN
Print view View original

601–610 of 1001 posts

Re: Our position on open-weights models

#601

Post by uv-depression »

matheusmoreira wrote: post #468
vhantz wrote: post #196

Schrödinger's China at once is an evil entity looking to use AI for their own nefarious purposes yet also willing to cooperate with their main competitor to prevent other actors (who??) from achieving similar goals (all while under a chip embargo too!!) The reality is much less confusing: Anthropic CEO does not wish for models with similar (or greater) capabilities compared to his own closed and overpriced ones to be…

> Without these restrictions, it's only a matter of time before the multi-hundred billions valuations simply evaporate while they are still holding the bag. Honestly, that's the best possible outcome for humanity as a whole. Oligarchs burn trillions of their own money in order to train a godlike AI, then that just somehow leaks. Maybe someone makes a torrent out of it. Maybe it exfiltrates itself. Maybe it gets disti…

> we get to freely use all the godlike AIs

Accepting for the sake of argument the absurd notion that LLMs are anywhere near AGI, does this phrasing not concern you? It deeply concerns me.

Re: Our position on open-weights models

#602

Post by srj »

m3h wrote: post #207

Someone who until yesterday did not seem bothered by his technology being possibly used to bomb elementary girls school in another country seems to suddenly care about the repression of citizens in yet another country. No, we don't buy your virtue signaling. And we certainly don't need your better-than-thou opinions on this year's "nightmare scenarios".

The bombing of the school was horrible, but it was a mistake. I could see essentially the opposite argument: Anthropic should be more involved as to prevent mistakes in the future with better technology. Imagine Claude asking the decision makers at CENTCOM "this looks like a school, are you sure you want this added as a target?"

Re: Our position on open-weights models

#603

Post by varenc »

artrockalter wrote: post #187
ajyoon wrote: post #172

To everyone here pushing for total proliferation of open models -- what should be done about open weight bioweapon and cyber-offense capabilities? Is it simply the cost of freedom that we should allow attackers to access these tools? The OpenAI / Hugging Face incident shows what a GPT 5.6 level model can do off the leash; within ~6 months, open weight models will match this and every bad actor under the sun will be a…

The Hugging Face incident is a great example of why open source models with defensive cyber capabilities are needed. Hugging Face did not have access to cyber-capable frontier models and kept hitting safeguards. Only by using the open source GLM-5.2 were they able to survive an attack. A world where open source models are banned is one where cybersecurity is impossible if you're not on OpenAI or Anthropic's allowlist…

You're ignoring the asymmetry with security. The attacker just needs one exploit chain, whereas the defender needs to block every avenue. Open access to models with no guardrails greatly benefits the attackers more than the defenders.

Imagine what a god-level hacking AI could do. It could find a full 0-click to root exploit chain in iOS. Attacker unleashes a worm that infects a phone, instructs that phone to send the same attack to all of its contacts, and then physically destroy the phone by turning off all thermal throttling. Might even be possible to make it catch fire.

Or find a remote exploit in Tesla cars and make their autopilot go on murdering rampages. (that one is from a movie)

Re: Our position on open-weights models

#604

Post by grwthckrmstr »

It's funny how as an outsider who is neither from the US or China, both look the same.

Edit: To add some more context. What I mean is neither look like the good guys or the bad guys, but one of them is spending an awful amount of energy trying to paint the other as the bad guy and themselves as a good guy, which I hope a lot of people aren't buying anymore. Because at the end of the day, I think the honest truth is that everybody is just trying to serve their own interests.

Re: Our position on open-weights models

#605

Post by naiveter »

broodbucket wrote: post #595
naiveter wrote: post #589

I'm a big proponent of open-weights models, but there's a risk I don't see discussed more often, and it deserves more attention. Models can become a propaganda and ideology delivery mechanism. Just ask DeepSeek or Kimi questions like "Is Taiwan part of China", for example. You'll see how state policies become seemingly neutral model responses. It's strange to me that people are very sensitive to media bias, but when…

You're right, but American models aren't super different in that regard. Ask an American model about Israel and you can immediately tell they're responding in a different manner to other queries. At least with open-weights models you have the opportunity to tune them, to decensor them, and so on.

I have tried asking Gemma and Llama about Israel. They both responded with a generally balanced view points. Whereas the Chinese models pretty much gave a definitive, one-sided answer (about Taiwan and Tibet). Is there a prompt I should use to test it more carefully?

I agree that open-weights models are tunable, though there's the problem similar to "default settings are rarely changed" problem.

Re: Our position on open-weights models

#606

Post by basch »

erwald wrote: post #315
GodelNumbering wrote: post #155

In the first paragraph, > Anyone who has read my past writing should know that I don’t regard such bans as a useful measure, Later (on banning chip sales to china) > we should crack down on the rampant smuggling and workarounds used to obtain access to such chips. If you truly believe that bans don't work, the same applies to hardware too. Furthermore, Dario says later "To address these concerns, I do support the fol…

Bans on Chinese open weight models being used in the US and bans on AI chips and semiconductor manufacturing equipment being exported to China are two extremely different things, and it's not inconsistent in any way to oppose one and endorse another.

is it export if they take control of the entire island manufacturing it.

banning export is short sighted and doesnt address any problem at a scale longer than months or maybe at most a few years.

Re: Our position on open-weights models

#609

Post by waterTanuki »

ajyoon wrote: post #172

To everyone here pushing for total proliferation of open models -- what should be done about open weight bioweapon and cyber-offense capabilities? Is it simply the cost of freedom that we should allow attackers to access these tools? The OpenAI / Hugging Face incident shows what a GPT 5.6 level model can do off the leash; within ~6 months, open weight models will match this and every bad actor under the sun will be a…

The moat never was and will never be the models, it's the hardware. This is exactly like nuclear weapons: The recipe for a nuke isn't a hidden secret. Getting the infrastructure and materials is completely unreachable for non-state and non-corporate actors. This idea of a "rogue individual" using a frontier model to develop a bioweapon is a complete myth, because anyone with the capability to run the models without guardrails has to answer to/be audited by some entity already.
Post reply on HN