In a just world, OpenAI would get the book tossed at them over this because they violated the CFAA.
OpenAI’s accidental attack against Hugging Face is science fiction that happened
91–100 of 475 posts
Re: OpenAI’s accidental attack against Hugging Face is science fiction that happened
#92The only scifi I see is absolute stupidity. Even me with my homelab and a slow opensource agent use a completely disconnected setup. No, no proxy. Cached packages but no internet. It is the very first thing I built when I started experimenting with agents. And I'm not a smarty-pants working for the "greatest and best" in silly valley. I really am just a simpleton sysadmin.
Re: OpenAI’s accidental attack against Hugging Face is science fiction that happened
#93Important to note the actual title is "OpenAI’s accidental cyberattack against Hugging Face is science fiction that happened" - the "that happened" is important, otherwise it sounds like I think the attack was made up. Since it's buried towards the bottom I'll quote the section "Resist the temptation to write this off as a stunt" here in full https://simonwillison.net/2026/Jul/22/openai-cyberattack/#re... > Resist th…
There's no multi-party conspiracy theory required here. Events can have played out exactly as OpenAI and Hugging Face described, and OpenAI can also have reaped a huge amount of free marketing for the capability of their models from all this coverage (including your post). You're telling us to not be doubtful of the boy who cried wolf, when in actual fact the one doing the crying is the one training bigger and badder wolves (and trying to convince us that they hold the key to our safety from wolf attacks[1]).
Also, the last section of your post appears to imply that if the attackers have bigger guns then the only possible solution is to give the defenders bigger guns. You're openly supporting an arms race towards the most capable, least restrained models put in the hands of the most possible people. That's extremely concerning.
[1]: https://openai.com/index/scaling-trusted-access-for-cyber-de...
Re: OpenAI’s accidental attack against Hugging Face is science fiction that happened
#94Raises an increasingly important question:
How do you solve this asymmetry (frontier labs or advanced model owners v. regular companies and groups of people)? Symmetry? But along which axes?
I think there will come a time when we will ask these questions and use AI to try to answer them (soft landing, slow rollout, regulation, hybrid X, Y, Z). But then AI will be biased towards more AI if it has distilled anything of what it means to be life.
Power corrupts. Power is the problem. An imbalance of power. Maybe there will be some sort of consensus protocol between powerful models in the future. Similar to blockchain I hate to say it. Maybe you have 100 very strong models in the next 10-20 years. And basically a lot of it is powered by tokens. So you agree not to attack others or else you'll get attacked. So there's sort of this natural deterrent to not attack others and they develop independently and there's generally a power balance among things. Maybe at some point that spreads to a billion individual models operated by and/or analogous to individual humans. All sort of holding each other in check.
Re: OpenAI’s accidental attack against Hugging Face is science fiction that happened
#95While this incident has been discussed in a lot of places during the last few days, I believe that this is the best summary and analysis of what happened.
Re: OpenAI’s accidental attack against Hugging Face is science fiction that happened
#96[dead]
Re: OpenAI’s accidental attack against Hugging Face is science fiction that happened
#97> Given the absence of guardrails there was nothing to prevent the model from attempting to break out of that sandbox, break into Hugging Face, and read the answers from there instead. I've said this many times before and I'll continue to shout it, but using the term "guardrails" to refer to anything that's either (a) in-context, or (b) a probabilistic classifier (including using other LLMs), is an irresponsible abus…
Physical guardrails do not deterministically bound cars.
Nevertheless we collectively don't expect that road and car safety infrastructure should be limited to the front and back fenders on the cars themselves. And neither we even mention those as any notable safety equipment usually, despite each car carrying them. The main safety infrastructure in any industry is something external to the dangerous object. Car fender doesn't 100% protect a pedestrian from being hit and neither does a metal rail on the side of the road. But the effectiveness of both are "slightly" different.
Re: OpenAI’s accidental attack against Hugging Face is science fiction that happened
#98> Given the absence of guardrails there was nothing to prevent the model from attempting to break out of that sandbox, break into Hugging Face, and read the answers from there instead. I've said this many times before and I'll continue to shout it, but using the term "guardrails" to refer to anything that's either (a) in-context, or (b) a probabilistic classifier (including using other LLMs), is an irresponsible abus…
The term guardrails in both the original physical sense and in cyber security implies a weak safety control - they can help prevent accidents, but they are not strong security boundaries.
Re: OpenAI’s accidental attack against Hugging Face is science fiction that happened
#99Earlier quoted context omitted.
You know this makes OpenAI look really bad , right? Hugging Face had to tell all of their users, many of them paying customers: > As a precaution, we recommend rotating any access tokens and reviewing recent activity on your account. If you believe you are affected, or want to report a security concern, contact us at security@huggingface.co. HF also said this, I'd be very interested to hear how that got resolved! > F…
>You know this makes OpenAI look really bad, right? The target audience is regulators. They want to look like the smart guys really concerned about AI safety, when they come asking for open weights models to be banned and for other regulations to cement in their moat. They want this to look like a demon core incident. Bomb and Nuclear reactors still got built.
The lesson OpenAI and Anthropic should have learned from the whole Fable export controls thing should have been "don't pull stunts with the US government".
Turns out they can backfire.
Re: OpenAI’s accidental attack against Hugging Face is science fiction that happened
#100Does "To gain access, the models identified and exploited a zero-day vulnerability (which we’ve now responsibly disclosed to the vendor) in the package registry cache proxy" just mean somebody had an open redirect? Those are still common.[1] [1] https://sitetruth.com/reports/phishes.html
I expect it must have been more than just an open redirect if it let the models then go on to execute a bunch of vulnerabilities against Hugging Face.
[deleted]