401–410 of 1001 posts
Just like how it was inevitable for SoTA LLMs to ignore copyright.
The actual challenge isn't how to prevent all these, but how stay on top.
And to stay on top it is inevitable to train unrestricted models. Anthropic is fighting windmills.
> Anthropic has never advocated for a ban on open-weights models. > All sufficiently capable models, open and closed, should go through mandatory safety testing. Yeah, this is anthropic advocating for a ban on open weight models. Who runs this test? What happens if this test is too costly or the administrator refuses to allow certain people to participate. This is exactly how the US has banned goods in the past, by r…
The goal is to make the safety tests cost $100M+, so that no one can release a model legally useable for a large portion of the world, unless they charge high enough prices, to the point where no one would use it, thus no competition.
Schrödinger's China at once is an evil entity looking to use AI for their own nefarious purposes yet also willing to cooperate with their main competitor to prevent other actors (who??) from achieving similar goals (all while under a chip embargo too!!) The reality is much less confusing: Anthropic CEO does not wish for models with similar (or greater) capabilities compared to his own closed and overpriced ones to be…
Yeah some real main character energy from Dario as usual. I'll never get why he thinks China would just sit there and let the US dominate them in AI when all it would take is a few of their boats blockading Taiwan to put a stop to it all.
Schrödinger's China at once is an evil entity looking to use AI for their own nefarious purposes yet also willing to cooperate with their main competitor to prevent other actors (who??) from achieving similar goals (all while under a chip embargo too!!) The reality is much less confusing: Anthropic CEO does not wish for models with similar (or greater) capabilities compared to his own closed and overpriced ones to be…
Yeah some real main character energy from Dario as usual. I'll never get why he thinks China would just sit there and let the US dominate them in AI when all it would take is a few of their boats blockading Taiwan to put a stop to it all.
The scary part very few are talking about is that every compute device is Turing complete. So everything from the phone in your pocket to a DGX Spark is a threat to national security now since, technically, every device can run any model (how well is not a question of concern when you start to argue hardware should be gated just the same as Dario likes to gate models). I mean, along these lines of thinking Linux should not be available to the masses! What if someone runs some code that's not approved by the benevolent dictator for life, Dario? People will say: that can't happen, but the reality is it already is. If everyone has reasonable access to compute to run models that are mostly capable comparative to burning Anthropic tokens, why wouldn't they? It's risk reduction and price protection. Yet we can't buy those systems because of future production already being purchased by these organizations.
But back to the models themselves... We played this game with Metasploit back in the day: many who had no clue claimed exploit tools should be regulated and only available for use by those blessed, illegal elsewhere (I believe the closest this got was the Wassenaar delegation in the US, but only through collateral inclusion of "cyber weapons "). Except in that timeframe the authors of these tools weren't advocating for protection. Today the world is fine, systems improved because of security FOSS tooling. The same thing will happen with LLMs. Unless, that is, Dario gets his way. I'm not a fan of Altman but I think he's standing back watching this play out knowing what Dario is doing: either he succeeds and OAI benefits or Dario ends up the Chicken Little of AI and Anthropic fails to launch (their IPO).
The reality is Dario is only doing this because this is a real risk to his business. China's constraints in building competitively have given them an advantage: they are doing more with less. And if you think that their distilling from US models was in any way anti-competitive or illegal, then I guess maybe "deal with it", much akin to Anthropic, Google and OAI's response around taking the (copyright) content in the first place with no repercussions.
People who don't work in the AI bubble don't care at all about any of these people. They could all be gone overnight and the world would continue to innovate, probably in a much more productive manner, without them.
To everyone here pushing for total proliferation of open models -- what should be done about open weight bioweapon and cyber-offense capabilities? Is it simply the cost of freedom that we should allow attackers to access these tools? The OpenAI / Hugging Face incident shows what a GPT 5.6 level model can do off the leash; within ~6 months, open weight models will match this and every bad actor under the sun will be a…
The software industry should be ashamed by the number of exploits that ai can find in software. It’s really an embarrassment. The software has to be built better.
It is really easy to have tunnel vision while coding. LLMs have a working memory with a capacity an order of magnitude greater than ours. I wouldn't trust an LLM to write the code, but at this point it is malpractice not to use one for review.
To everyone here pushing for total proliferation of open models -- what should be done about open weight bioweapon and cyber-offense capabilities? Is it simply the cost of freedom that we should allow attackers to access these tools? The OpenAI / Hugging Face incident shows what a GPT 5.6 level model can do off the leash; within ~6 months, open weight models will match this and every bad actor under the sun will be a…
Like the others here I know almost nothing about bio weapons, but I think perhaps the fact that smallpox's genome sequence has publicly available in scientific databases like GenBank for 30 years is relevant. That horse bolted a long time ago.
> Anthropic has never advocated for a ban on open-weights models. > All sufficiently capable models, open and closed, should go through mandatory safety testing. Yeah, this is anthropic advocating for a ban on open weight models. Who runs this test? What happens if this test is too costly or the administrator refuses to allow certain people to participate. This is exactly how the US has banned goods in the past, by r…
Exactly correct. This technique has been used again and again to discourage competition. I was asked was they could have done to encourage competition and I said, "Lobby to make the entity that provided the model unwaivably liable for consequential and incidental damages of its use." That way people who built models pay the price for the lack of safety testing. We both agreed that would probably kill most of the AI m…