Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Our position on open-weights models

anthropic.com

Post reply on HN
Print view View original

871–880 of 1001 posts

Re: Our position on open-weights models

#871

Post by Diogenesian »

alevskaya wrote: post #647
ajyoon wrote: post #172

To everyone here pushing for total proliferation of open models -- what should be done about open weight bioweapon and cyber-offense capabilities? Is it simply the cost of freedom that we should allow attackers to access these tools? The OpenAI / Hugging Face incident shows what a GPT 5.6 level model can do off the leash; within ~6 months, open weight models will match this and every bad actor under the sun will be a…

I was a genetic engineer for ~20 years and have worked on frontier LLMs for the last 8. I used to engineer viral vectors and studied how to evade human immune systems for gene therapies... The biorisk scenarios that the AI safety folks flog are fever-dreamed fantasies that have only the most tenuous connection to biological reality. As someone who cares about the real bio-risks of natural pathogens, I get pretty tire…

It has always been ridiculous to suppose that some organization spent $500m on a microbiology / genetics lab, but ran out of money for scientists, so they have to ask Claude what to do. Or OTOH to suppose a guy in his garage set up a weapons-grade CRISPR lab without anybody noticing.

Re: Our position on open-weights models

#873

Post by lukewarm707 »

sfink wrote: post #512
lukewarm707 wrote: post #312

dario, most of the world wants CHINA to win because the USA is the bad guy. you should be worried about the USA having these models.

Anthropic is a US company, so it is lobbying the US to enact regulations to serve both its interests and the interest of safety. What else should it be doing? I imagine Anthropic (or at least many of the people who work there) are fully aware that USA is the bad guy right now, but that doesn't change what levers they have and do not have available.

it's easy to forget. openai and anthropic, were at one time more like foundation-style nonprofits interested in open source research. the idea was to benefit humanity, not US interests and not company interests.

over time they became closed source, for profit, US military contractors, and the clients of expensive political lobbyists.

anthropic once said it was their identity to stop scaling before the current model capability; they rewrote the responsible scaling policy so that they could continue scaling. openai said in its founding document that it would be unconstrained by return on investment; it restructed into the for profit setup. deepmind signed documents when it was aquired by google stating that its ai would not be used for military purposes; they rewrote these commitments to sell gemini to the us military. it wasn't like this!

it did not have to be this way. even though everyone will IPO and become yet more accountable to shareholders, there is still time.

Re: Our position on open-weights models

#874

Post by ianm218 »

Stitch4223 wrote: post #646

$regulatory_capture https://en.wikipedia.org/wiki/Regulatory_capture On processing power, copyright, and capability. I like to think of it as a knives factory. Anthropic knives are crafted with superior technology, uniquely shaped to perfection, and safe to operate. As seen on TV. Millions are hurt by knives each day. Every household has tons of them, making everyone a potential mouth-foaming murderer 24/7. But not w…

Where does the bio risk he mentions fall in this category? It feels quite plausible in a year or two to have closed loop labs with very modest resources.

Doesn’t mean that you have to think only Anthropic should be able to make bioweapons or whatever, but it just feels like this ignores the risk

Re: Our position on open-weights models

#875

Post by ianm218 »

jameson wrote: post #83

The concerns are legitimate but the proposals are nothing more than a stopgap solution. If US wants to maintain engineering superiority, we needs to invest in it -- education, research and infrastructure. Bring in top researchers across the globe and not make it harder. China is building infrastructure for the future generations and investing in growth sectors while the US is cutting of university grants and spending…

I don’t disagree but the timeline for Dario’s concerns are quite short. Open weight models could feasibly lead to serious bio safety concerns in the next year or so while the US’ current issues are going to take a long time to work through.

China is also financially repressing their people to build often useless infrastructure projects to be fair they are also not the good guys.

Re: Our position on open-weights models

#876

Post by ianm218 »

nxtfari wrote: post #394

Surprisingly incoherent for Anthropic and Dario (cue peanut gallery — “always has been!” No, I don’t think so. I think this is new). It seems to me like there is just no good answer to how one could possibly stop open weight models from being used for nefarious purposes. How are you going to enforce guardrails on open source? The only way is to turn the USA into a 1984-type totalitarian surveillance state (even more…

I think you need to be more clear in your argument.

Training a decent open weight model takes atleast millions of dollars and they are all associated with real people and companies, almost exclusively in the US and China. So just make normal open weigjt go through bio testing before release and that gets you a lot of the way there.

There is more problems like finetuning but you need to start somewhere.

Re: Our position on open-weights models

#877

Post by regexorcist »

paxys wrote: post #81

Hate to break it to you Dario but the way things stand right now the world at large trusts the Chinese establishment a lot more than the American one.

He is well aware. This is just another pathetic attempt at being handed a monopoly based on fear-mongering and protectionism.

Re: Our position on open-weights models

#878

Post by ianm218 »

Laurel1234 wrote: post #862
SubiculumCode wrote: post #632

Anthropic/Dario getting a lot of hate in this thread, but the argument he makes is rational if you believe for one second that AI is important to national security and that it can, or will soon be incredibly dangerous is misused. People pointing out that this position benefits his company might be overly reductionist in how they think the world works. I believe Dario and Anthropic about their belief that AI safety is…

Dario is a fascist who wholeheartedly believes Yankee exceptionalism and its divine right to brutalize anyone in order to achieve its goals. Anthropic's models have helped bomb innocents over and over and over again, and will continue to do so. If you think he gives half a shit about safety you've had a dozen lobotomies too many and should reconsider every decision you've ever made.

Can you explain how he is a fascist?

Re: Our position on open-weights models

#879

Post by ianm218 »

abotsis wrote: post #643

China releasing open weight models, powerful or not, does nothing to prevent their development of models they’ll use for evil. Nor does it stop someone from abliterating a non-Chinese model and using it for evil. Sorry- I don’t see any other reason aside from Anthropic protecting their own interests.

What are you arguing against? He doesn’t advocate for a ban on Chinese open source or non Chinese open source.

Re: Our position on open-weights models

#880

Post by ianm218 »

pianopatrick wrote: post #116

I don't really see the link between use of AI and military superiority. My current understanding is a lot of current US military problems are due to rare earths supply chains. I don't see how AI would either help or hurt with that.

Here is an example from the conflict in Ukraine.

Ukraine is currently leading a long strikes campaign. You could prompt the right model to A. Find all the refineries in Russia B. For each refinery annotate all the equipment, find the most value-able C. Make a fluid model of the refinery and figure out which components most likely to set off a chain reaction if hit.

Add stuff like drone swarms monitoring the front line etc on top of that.

Supply chain issues like rare earths are real… but not really related.

Post reply on HN