Anthropic's fall from grace and mindshare seems rather accelerated.
I wonder if these rapid movements are going to be the norm now. I imagine there would be angry investors if this sort of thing happened with a public company.
Our position on open-weights models
261–270 of 1001 posts
Re: Our position on open-weights models
#262You can put lipstick on a pig, it'll still be a pig "Anthropic has never advocated for a ban on open-weights models." --- "We should crack down on industrial-scale distillation operations" "All sufficiently capable models, open and closed, should go through mandatory safety testing" These are in tension with advocating for open weight models. Not direct but enough that it calls into question the first statement. What…
> an advanced, closed, American AI model hacked another company. The only defense was open-source AI from China. HuggingFace did not seek access to Claude Mythos or OpenAI's equivalent program. They probably could have had access to these models for defensive purposes if they'd done it properly. > these statements are counter-factual. The OpenAI incident is a single example. You're massively overgeneralizing. You can…
"HuggingFace did not seek access to Claude Mythos or OpenAI's equivalent program. They probably could have had access to these models for defensive purposes if they'd done it properly."
HF released a statement and made it clear a closed source model specialized in cyber security refused them. They stated they had to use open source. What model is specialized in cyber security, closed, and frequently denies users access other than Mythos/Fable and 5.5Cyber? If not these two, what was HF referring to? It sounds like you have a source, I would like to read it.
fwipsy is right, cnbc has a story on this. they only had fable. I still think this is horrible for closed source, get on a list or else, but i was wrong
"You can't refute an entire class of possible outcomes based on a single event where it went the other way."
But Dario can dream up and entire class of outcomes based on the zero events that have never gone his way? Convenient.
The OpenAI incident is singular and HF was clear, it went exactly how I wrote it: a closed source American AI decided to perform corporate espionage and the only tool available was open source AI from China
"I tend to agree that model capabilities will favor defense over attack, but I think there will be a lot of disruption before that equilibrium is reached. If cybercriminals or state-sponsored actors are able to scale up attacks quickly, many orgs with less sophisticated defenses will be caught by surprise."
We literally just saw an advanced model from openAI commit a cyber crime. I can't take hypotheticals that ignore reality seriously and it shouldn't be lauded as some higher form of thought
Re: Our position on open-weights models
#263> Anthropic has never advocated for a ban on open-weights models. > All sufficiently capable models, open and closed, should go through mandatory safety testing. Yeah, this is anthropic advocating for a ban on open weight models. Who runs this test? What happens if this test is too costly or the administrator refuses to allow certain people to participate. This is exactly how the US has banned goods in the past, by r…
A government agency tests all medications, why not models?
Don’t think government controlling AI is a good idea.
Not sure if they have an understanding of AI in the first place. Secondly, even though AI companies claim that they have achieved AI that needs to be heavily monitored (maybe for PR purposes), I’m not sure if that is true. Sam Altman said the same things about GPT-4 that Anthropic is now claiming about Mythos.
Government control will be a good idea once we start approaching AI that is actually destructive.
Also even if we decide to put controls in place what is the guarantee that china will do the same, specially for a model which is not actually destructive.
Re: Our position on open-weights models
#264> Anthropic has never advocated for a ban on open-weights models. > All sufficiently capable models, open and closed, should go through mandatory safety testing. Yeah, this is anthropic advocating for a ban on open weight models. Who runs this test? What happens if this test is too costly or the administrator refuses to allow certain people to participate. This is exactly how the US has banned goods in the past, by r…
Yeah, seems pretty likely. Anthropic will make the case that their models should be evaluated with the safety layer in front, because that is the only way the model is available whereas open weight models need to pass the same test just on the weights. The economic implications will be rather large, but in terms of security it seems inconsequential. The most compelling argument would be that by limiting the use of op…
Makes sense why OpenAIs little "hacking" stunt was published last week
Re: Our position on open-weights models
#265It is obviously not going to be until some really bad series of cyberattacks or a chemical/bioweapon attack before anyone takes regulation of models seriously. They are _obviously_ (please convince me otherwise) going to be capable of carrying these terrible things out almost completely autonomously at some point in the near future, in potentially clever ways. Therefore we must, at some point, ban or heavily regulate…
Why would you assume that the same capability could not be used to harden systems?
Re: Our position on open-weights models
#266The distillation commentary is really crazy coming from a company that stole all it's training material.
Re: Our position on open-weights models
#267Woof, talk about self aggrandizing.
Re: Our position on open-weights models
#268Anthropic starting to sound like Microsoft from the 1990s.
Re: Our position on open-weights models
#269> All sufficiently capable models, open and closed, should go through mandatory safety testing What happens if a model fails the test? Surely one can use Kimi K3 for evil, somehow or other. What now? "Mandatory safety testing" implies consequences for failing, yet Dario has nothing to say about what the consequences should be. He says he doesn't advocate a ban but it's hard to imagine what his alternative would be if…
If a model fails the test, it should be banned. He is not advocating a ban of open-weight models. He is advocating a ban of models that fail mandatory safety testing. Seems reasonable and straightforward.
Re: Our position on open-weights models
#270Amazing how this company went from having such goodwill to hardly any. At the end of the day they want to make as much money as possible. Anthropic will have to lose a ton of their profitability if they compete with open source. I see them moving into the application layer, which they’ve already started doing. It’s clear open models are the future for the vast majority of use cases that don’t need frontier capabilities.