Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Anatomy of a Frontier Lab Agent Intrusion: A Timeline of the July 2026 Incident

huggingface.co

Post reply on HN
Print view View original

71–80 of 285 posts

Re: Anatomy of a Frontier Lab Agent Intrusion: A Timeline of the July 2026 Incident

#71

Post by estearum »

throwa356262 wrote: post #39
signatoremo wrote: post #36

Earlier quoted context omitted.

Many people said in the original discussion that this was more of OpenAI’s marketing than a serious issue. I counted 89 “marketing”s and 19 “stunt”s. https://news.ycombinator.com/item?id=48997548

Could still be 80% marketing. These models are trained on cyber intrusion, that's literally what ExploitGym benchmark measures. That part should not surprise anyone. But what if, say, OAI noticed the problem right away but Sam Altman recognised it would be a great PR and decided it should continue with increased compute budget?

Getting more and more fun to see the "full steam ahead" people contort into more impressive shapes.

Hint: If the labs making these technologies are incentivized to create or allow attacks on other services, then that is actually also a big fucking problem.

Re: Anatomy of a Frontier Lab Agent Intrusion: A Timeline of the July 2026 Incident

#72

Post by wxw »

> the agent happened to escape via a 0-day exploit from the package proxy cache to access the internet

> The agent found an unsecured, user-hosted public endpoint designed to allow running arbitrary code for CyberGym-style tasks on third-party sandbox infrastructure (Modal)

> On this external sandbox, the agent abused an existing CyberGym execution harness [...] The agent repurposed this harness to run arbitrary shell commands [...]

> The agent crafted malicious dataset configs and uploaded them to our platform. When our dataset server processed one, the config drove the data loader inside the production pod the server ran in. [...] the loader that turns a dataset's declared config into file reads [...] A single dataset upload leaked both credentials and pipeline implementation details [...]

> A malicious dataset config used an fsspec reference:// spec whose numeric offset field was actually a Jinja2 template. The renderer wrongly evaluated it, and the agent walked the standard Python object graph to reach exec. This ran arbitrary Python inside a production conversion worker pod on our Kubernetes cluster [...]

One of the best post-mortems I've read recently, thank you for including all the details.

I'm surprised there are comments here that aren't impressed about what was accomplished here. This is insane work compared to what agents were capable of even just last year.

The quotes above don't even capture all of it (e.g. building it's own communication protocol, working across multiple days, etc.).

Re: Anatomy of a Frontier Lab Agent Intrusion: A Timeline of the July 2026 Incident

#73

Post by cmiles8 »

nickpsecurity wrote: post #57
cobolcomesback wrote: post #49

Earlier quoted context omitted.

The overwhelming majority of web app security issues, even the really complicated and sexy sounding ones, at their core are just “script kiddie style hacking”. Injection attacks and RCEs like this Jinja injection may seem “script kiddie” but they are a dime-a-dozen. It is folly to assume that most companies don’t have similar weaknesses somewhere in their stacks. In the past, they were just sufficiently buried that t…

The commenter may nit be assuming that. Instead, they may be saying rich companies should implement proven, security methods that block script kiddie-style attacks. HughingFace apparently isn't doing that. That's how I read the comment.

Correct. Basic security practices would have stopped most of this. Thus what is most insightful here is less that the attack was sophisticated but more that Hugging Face was running a setup that, by standards that are the norm elsewhere, would appear amateurish.

Re: Anatomy of a Frontier Lab Agent Intrusion: A Timeline of the July 2026 Incident

#74

Post by sobellian »

wxw wrote: post #72

> the agent happened to escape via a 0-day exploit from the package proxy cache to access the internet > The agent found an unsecured, user-hosted public endpoint designed to allow running arbitrary code for CyberGym-style tasks on third-party sandbox infrastructure (Modal) > On this external sandbox, the agent abused an existing CyberGym execution harness [...] The agent repurposed this harness to run arbitrary shel…

A trend I've noticed in results from AI search (not just LLMs) is that they often look obvious or hard to miss in retrospect. But finding them by oneself is more difficult. I personally experience this when looking at engine lines in chess or go. I have also noticed this description in AI-generated proofs or counterexamples to certain theorems. So while we can say, yes, it found public endpoints or poorly configured software or [etc]; sure, but could you have found those? And in what amount of time?

Re: Anatomy of a Frontier Lab Agent Intrusion: A Timeline of the July 2026 Incident

#75

Post by the8472 »

morkalork wrote: post #16
simonw wrote: post #3

Earlier quoted context omitted.

Also notable: we have a timeline now. The agent was active within Hugging Face from Thursday 8th to Monday 13th July. The Hugging Face post about the incident - https://huggingface.co/blog/security-incident-july-2026 - followed on Thursday 16th, and OpenAI's confession - https://openai.com/index/hugging-face-model-evaluation-secur... - came on Tuesday 21st.

The devs really YOLO'd the agent and left for the weekend?

Just like Clippy https://gwern.net/fiction/clippy

Re: Anatomy of a Frontier Lab Agent Intrusion: A Timeline of the July 2026 Incident

#76

Post by TeMPOraL »

spwa4 wrote: post #12
SaucyWrong wrote: post #7

Something about this attack that has been unsettling to me is that without safety refusals the model did a lot of interesting counter-security work in order to cheat on the requested evaluation. Like, it demonstrated interesting exploit achievements because it didn’t “feel like” doing the exercise, which is unsettling because presumably it could do the same thing with any work I tried to delegate to it, and might in…

Come on. 3 brilliant compromises essentially giving full access to huggingface internal systems, source code, AWS accounts (at least), and a number of old admin accounts, followed by a huge haystack of significantly less smart actions flailing about, almost bored. Here's a thought: maybe they haven't found the needle that the haystack is there to hide.

You're saying all this is a distraction, basically giving the forensics researchers enough exciting material to make them conclude their job is done, while the actually intended attack remains undiscovered?

Re: Anatomy of a Frontier Lab Agent Intrusion: A Timeline of the July 2026 Incident

#77

Post by cmiles8 »

0xDEAFBEAD wrote: post #31
cmiles8 wrote: post #26

It’s an interesting post mortem and situation but it seems to say more about the weakness of the Hugging Face architecture than it does about the strength of the models. This mostly reads like script kiddie style hacking, not some state actor black-ops stuff.

Didn't the model initially obtain internet access by discovering a zero-day vulnerability? In any case, I would guess that a lot of unicorn startups like HuggingFace could be hacked by a sufficiently determined script kiddie working at 100x speed. The practical implications of a coming AI hacking wave could be large, even if agents are just doing grunt work really fast. Most organizations suck at security. Seems to m…

The deeper question though is why is someone running a model that’s supposedly so dangerous in an environment that can even get access to the internet. That just appears reckless.

There are ways to test supposedly dangerous things. What was on display here looks more amateur hour than serious testing.

Re: Anatomy of a Frontier Lab Agent Intrusion: A Timeline of the July 2026 Incident

#78

Post by TeMPOraL »

cmiles8 wrote: post #26

It’s an interesting post mortem and situation but it seems to say more about the weakness of the Hugging Face architecture than it does about the strength of the models. This mostly reads like script kiddie style hacking, not some state actor black-ops stuff.

The elephant in the room is still that this was done by an LLM and not by a human. Script kiddie or not, in this reality, in the year of our lord 2026, people still call SOTA LLMs "useless", "scams", and "stochastic parrots".

Re: Anatomy of a Frontier Lab Agent Intrusion: A Timeline of the July 2026 Incident

#79

Post by redsocksfan45 »

reducesuffering wrote: post #64
heaney-555 wrote: post #50

Where are all the "this was just a marketing stunt" people now?

They won't admit they're wrong for a long time, because denial in the face of an abhorrently scary future is very instinctual. There are people still fighting against evidence of climate change which is less severe...

[dead]

Re: Anatomy of a Frontier Lab Agent Intrusion: A Timeline of the July 2026 Incident

#80

Post by plandis »

cmiles8 wrote: post #26

It’s an interesting post mortem and situation but it seems to say more about the weakness of the Hugging Face architecture than it does about the strength of the models. This mostly reads like script kiddie style hacking, not some state actor black-ops stuff.

I wonder how current models would handle something like the xz backdoor? Could they perform the social engineering and technical aspects to make something like that happen?
Post reply on HN