Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI’s accidental attack against Hugging Face is science fiction that happened

simonwillison.net

Post reply on HN
Print view View original

101–110 of 475 posts

Re: OpenAI’s accidental attack against Hugging Face is science fiction that happened

#101

Post by sandos »

Teever wrote: post #48

It absolutely is science-fiction. This recent event is more or less the plot-line to my favourite X-Files episode named Killswitch which was written by William Gibson.[0] This episode also features one of the coolest intros of any television episode ever[1] We really are rapidly approaching the cyberpunk dystopia that people like Phillip K. Dick and William Gibson wrote about. More than ever we need to be consulting…

Ah, yes, I started thinking about William Gibsons work right away when thinking about people just using powerful AIs, but being hamstrung by corporations.

He writes a lot of about basically DoS:ing the legal system, or if it was patents system, with a storm of litigation using automation or AI. Also not impossible in the future.

Re: OpenAI’s accidental attack against Hugging Face is science fiction that happened

#102

Post by joshka »

simonw wrote: post #65
Animats wrote: post #63

Earlier quoted context omitted.

If it lets you do an arbitrary HTTP GET on a URL sent as a parameter to the main URL, you've escaped the sandbox rules.

How do you then use GET requests to create a malicious dataset package and publish that to Hugging Face in order to exploit their package building infrastructure?

There's no reason to think that a tool that can find an 0-day in a repo cache can't work out how to make that host send a post request rather than a get request once it has its keys and is able to get it to make arbitrary web calls.

Re: OpenAI’s accidental attack against Hugging Face is science fiction that happened

#103

Post by joshka »

simonw wrote: post #33
Animats wrote: post #27

Does "To gain access, the models identified and exploited a zero-day vulnerability (which we’ve now responsibly disclosed to the vendor) in the package registry cache proxy" just mean somebody had an open redirect? Those are still common.[1] [1] https://sitetruth.com/reports/phishes.html

I expect it must have been more than just an open redirect if it let the models then go on to execute a bunch of vulnerabilities against Hugging Face.

I wonder if it was this bug fixed july 14 in sonatype:

https://support.sonatype.com/hc/en-us/articles/5316501964136...

pure speculation here - no non public info

Re: OpenAI’s accidental attack against Hugging Face is science fiction that happened

#104

Post by simonw »

joshka wrote: post #103
simonw wrote: post #33

Earlier quoted context omitted.

I expect it must have been more than just an open redirect if it let the models then go on to execute a bunch of vulnerabilities against Hugging Face.

I wonder if it was this bug fixed july 14 in sonatype: https://support.sonatype.com/hc/en-us/articles/5316501964136... pure speculation here - no non public info

I had a look at that one and several other Nexus Repository issues that share the July 14th date. I don't think they're it, for two reasons:

Firstly, they all credit named individuals who don't seem to have a relationship with OpenAI - this one credits e0x1337 for example and https://hackerone.com/e0x1337?type=user links to https://www.linkedin.com/in/aimanharith

Secondly, the 14th of July feels too early. HF reported the incident on the 16th and OpenAI only responded in the 21st. These issues are all patched, so they should have been reported days or weeks before the 14th.

Re: OpenAI’s accidental attack against Hugging Face is science fiction that happened

#105

Post by simonw »

joshka wrote: post #102
simonw wrote: post #65

Earlier quoted context omitted.

How do you then use GET requests to create a malicious dataset package and publish that to Hugging Face in order to exploit their package building infrastructure?

There's no reason to think that a tool that can find an 0-day in a repo cache can't work out how to make that host send a post request rather than a get request once it has its keys and is able to get it to make arbitrary web calls.

If the vulnerability is purely an open redirect that doesn't work for me.

Clearly there was a hole in the software but I don't think open redirect is the likely initial problem.

Hopefully we will find out for sure in a few days.

Re: OpenAI’s accidental attack against Hugging Face is science fiction that happened

#106

Post by simonw »

Wurdan wrote: post #93
simonw wrote: post #10

Important to note the actual title is "OpenAI’s accidental cyberattack against Hugging Face is science fiction that happened" - the "that happened" is important, otherwise it sounds like I think the attack was made up. Since it's buried towards the bottom I'll quote the section "Resist the temptation to write this off as a stunt" here in full https://simonwillison.net/2026/Jul/22/openai-cyberattack/#re... > Resist th…

There's no multi-party conspiracy theory required here. Events can have played out exactly as OpenAI and Hugging Face described, and OpenAI can also have reaped a huge amount of free marketing for the capability of their models from all this coverage (including your post). You're telling us to not be doubtful of the boy who cried wolf, when in actual fact the one doing the crying is the one training bigger and badder…

I had real trouble deciding how to end this piece.

There's no easy answer here. All of the options are bad in different ways!

As a builder of software, I want access to the best possible tools to help me keep that software secure.

As a user of software, I want my software to be secure and I don't want bad actors to be able to access tool to help them exploit it.

Is the only answer here to have the AI labs make decisions over who gets access to the tools? What if they make mistakes in those decisions?

None of the options look good to me. I don't know what we should do here.

My hunch is that the open weight models are already forcing our hand. The dangerous capabilities are coming to everyone.

Re: OpenAI’s accidental attack against Hugging Face is science fiction that happened

#107

Post by rurban »

spwa4 wrote: post #90
mnicky wrote: post #86

I think points that deserve more attention in the current public discourse are: - This should be a huge wakeup call for everybody. - We are lucky that it wasn't a case of an agent running a virology lab benchmark that decides to hack a lab and tries to synthesize something. - It also shows apparent lack of competence and oversight from OpenAI: how is it that they didn't quickly find that agent is breaking the sandbox…

It's a fake PR issue. It's hardly the first time this happens, but of course OpenAI, with its IPO now more in doubt than ever, had to claim this (and, once again, I have trouble believing Sam Altman choosing this: this could lead to OpenAI getting regulated, which has at least as much potential to lower their IPO price as to raise it). But there have been messages about LLMs, especially coding agents, "grabbing root"…

Nonsense. Huggingface reported it to police!

Re: OpenAI’s accidental attack against Hugging Face is science fiction that happened

#108

Post by rurban »

spwa4 wrote: post #90
mnicky wrote: post #86

I think points that deserve more attention in the current public discourse are: - This should be a huge wakeup call for everybody. - We are lucky that it wasn't a case of an agent running a virology lab benchmark that decides to hack a lab and tries to synthesize something. - It also shows apparent lack of competence and oversight from OpenAI: how is it that they didn't quickly find that agent is breaking the sandbox…

It's a fake PR issue. It's hardly the first time this happens, but of course OpenAI, with its IPO now more in doubt than ever, had to claim this (and, once again, I have trouble believing Sam Altman choosing this: this could lead to OpenAI getting regulated, which has at least as much potential to lower their IPO price as to raise it). But there have been messages about LLMs, especially coding agents, "grabbing root"…

Nonsense. Hugging face reported it to police.

Also very likely that it actually happened as reported. My own agents always trying to "cheat", eg. by fixing tests instead of fixing the code. That's normal operation, unless you tell it ("harness"), not to do so.

Re: OpenAI’s accidental attack against Hugging Face is science fiction that happened

#109

Post by joshka »

simonw wrote: post #104
joshka wrote: post #103

Earlier quoted context omitted.

I wonder if it was this bug fixed july 14 in sonatype: https://support.sonatype.com/hc/en-us/articles/5316501964136... pure speculation here - no non public info

I had a look at that one and several other Nexus Repository issues that share the July 14th date. I don't think they're it, for two reasons: Firstly, they all credit named individuals who don't seem to have a relationship with OpenAI - this one credits e0x1337 for example and https://hackerone.com/e0x1337?type=user links to https://www.linkedin.com/in/aimanharith Secondly, the 14th of July feels too early. HF reporte…

The other suspicion I had was jfrog artifactory (you mentioned OpenAI use it in a blog post in Jan). I accidentally conflated that with Nexus 14 july CVE. I think that's probably more likely.

I suspect it may be possible to throw a local model (same as hf did :D) at jfrog and find the exact mechanism in a small amount of hours.

Re: OpenAI’s accidental attack against Hugging Face is science fiction that happened

#110

Post by sillyfluke »

rurban wrote: post #108
spwa4 wrote: post #90

Earlier quoted context omitted.

It's a fake PR issue. It's hardly the first time this happens, but of course OpenAI, with its IPO now more in doubt than ever, had to claim this (and, once again, I have trouble believing Sam Altman choosing this: this could lead to OpenAI getting regulated, which has at least as much potential to lower their IPO price as to raise it). But there have been messages about LLMs, especially coding agents, "grabbing root"…

Nonsense. Hugging face reported it to police. Also very likely that it actually happened as reported. My own agents always trying to "cheat", eg. by fixing tests instead of fixing the code. That's normal operation, unless you tell it ("harness"), not to do so.

In the parent's comment I initially read "PR" as "Public Relations" not problem report since their comment is about OpenAI and does not directly accuse Hugging Face except for that phrase. But it's still amiguous to me who's PR they are actually talking about. A good faith reading given the rest of the comment leads me to assume they are talking about OpenAI, not Higgins Face.

I did actually search Hugging Face with police in quotes and found no articles containg the word police but that might be a ddg thing. Then I checked Hugging Face's report and they specifically use the term "law enforcement" not police, as is to be expected I guess. So that checks out.

Can you find any info on the police investigation by the way? At the least, OpenAI should be investigated for potential criminal negligence, right?

Right?

Post reply on HN