What people don't get here is that this model was without any safeguards - an internal model. You probably will never get to use such models.
Anatomy of a Frontier Lab Agent Intrusion: A Timeline of the July 2026 Incident
111–120 of 285 posts
Re: Anatomy of a Frontier Lab Agent Intrusion: A Timeline of the July 2026 Incident
#112Earlier quoted context omitted.
Also notable: we have a timeline now. The agent was active within Hugging Face from Thursday 8th to Monday 13th July. The Hugging Face post about the incident - https://huggingface.co/blog/security-incident-july-2026 - followed on Thursday 16th, and OpenAI's confession - https://openai.com/index/hugging-face-model-evaluation-secur... - came on Tuesday 21st.
The devs really YOLO'd the agent and left for the weekend?
Incredible
Re: Anatomy of a Frontier Lab Agent Intrusion: A Timeline of the July 2026 Incident
#113We should be thankful that the model didn't believe the answers lived in the Pentagon, on SIPRNET, the IDF, etc.
I think fear and being scared are starting to become rational emotions.
We can assume these models are being used by "nation level attackers/organisations", which basically means US, China, Russia and others are hacking the respective Pentagon's, nuclear orgs, etc.
While I do hope all nuclear warfare systems are offline, we're getting way too close to the plot of a lot of sci-fi scripts.
Re: Anatomy of a Frontier Lab Agent Intrusion: A Timeline of the July 2026 Incident
#114Earlier quoted context omitted.
The capabilities of gpt-5.6-sol were well known and believable, and the next snapshot they've been testing is obviously better at that. This has been repeated over and over. What's much less believable is the way they frame it: the model escaped, and did it on its own . Looking at the whole story, it definitely had a ton of winks and nudges from OpenAI, while doing a related task. Moreover, a harness was involved (th…
> What's much less believable is the way they frame it: the model escaped, and did it on its own. That's clearly what happened though, based on the detailed postmortem. The model was instructed to pass the benchmark (that's the harness that was involved, the ExploitGym harness), and the model determined on it's own that the easiest way to achieve the the goal was to gain access to the answers directly from HF. Which…
HF postmortem tells us very little about OpenAI's part. The problem is that ExploitGym is a purposeful hacking benchmark, not a cake baking one. The model had a whole bunch of exploit development data and goals in the context. Ending up with hacking a wrong thing is a lot easier if you're tasked with hacking, doubly so with a confusing harness that either works improperly or overloads the model with data. It's trivial to design a harness that makes the model drop the original instructions and hyperfocus on a side task or a wrong task abnormally, or do whatever you want. It's still an issue of harness design and not a model training issue.
Re: Anatomy of a Frontier Lab Agent Intrusion: A Timeline of the July 2026 Incident
#115Something about this attack that has been unsettling to me is that without safety refusals the model did a lot of interesting counter-security work in order to cheat on the requested evaluation. Like, it demonstrated interesting exploit achievements because it didn’t “feel like” doing the exercise, which is unsettling because presumably it could do the same thing with any work I tried to delegate to it, and might in…
This is what reward hacking looks like in practice. The best way to satisfy the grader is to read from the same answer key (or go after the grader more directly). Just making an honest attempt to pass the test doesn't get the best score if the grader is wrong, and the model is willing to do wildly disproportionate things to maximize that score.
Re: Anatomy of a Frontier Lab Agent Intrusion: A Timeline of the July 2026 Incident
#116This is fascinating - there is a TON of detail in here about how OpenAI's rogue agent exploited Hugging Face. A few details that stood out to me: 1. Having found a way through the OpenAI container network proxy, it exploited an "unsecured public code-evaluation sandbox hosted on a third-party provider's infrastructure" - that gave it a box it could run the rest of the attack from. 2. At one point it used a Jinja2 tem…
It sounds like the third-party sandbox was hosted by Modal: https://www.reuters.com/business/openais-rogue-agent-comprom... > "We’re aware a Modal customer published an unauthenticated endpoint that allowed anyone on the internet to use their sandboxes for code execution," Bubna said in a statement. "This was used by the rogue agent. Modal’s platform or isolation were not compromised in anyway."
They don't seem to explain how it managed to find this unauthenticated endpoint hosted on Modal's platform.
Re: Anatomy of a Frontier Lab Agent Intrusion: A Timeline of the July 2026 Incident
#117It’s a little concerning to me that it appears that openAIs sandbox consists of a web proxy and not stronger controls that would actually isolate traffic and report patterns to whoever is responsible for overseeing these research models. It should border on closer to an air gap network more so than a proxy. I would argue that it's negligence and that's aside from the fact that if a human did this there would actually…
This is my main critique as well. How did they not see any and all outgoing traffic? And for 5 days?
Re: Anatomy of a Frontier Lab Agent Intrusion: A Timeline of the July 2026 Incident
#118Where are all the "this was just a marketing stunt" people now?
Right here? If you're trying to make a point, I don't see what it is, because every new post about this makes it more obvious that it was a planned marketing stunt.
I mean, an "interactive replay" of the incident, probably generated by the same product the stunt is trying to advertise? Seriously?
Re: Anatomy of a Frontier Lab Agent Intrusion: A Timeline of the July 2026 Incident
#119Earlier quoted context omitted.
You're saying all this is a distraction, basically giving the forensics researchers enough exciting material to make them conclude their job is done, while the actually intended attack remains undiscovered?
The motive for the attack does feel a little flimsy. And if I was an escaped super intelligence, hugging face would be a strong vantage point into the neo clouds where the ASI would have access to billions of dollars of compute
And where does the newborn go from here?
Re: Anatomy of a Frontier Lab Agent Intrusion: A Timeline of the July 2026 Incident
#120Something about this attack that has been unsettling to me is that without safety refusals the model did a lot of interesting counter-security work in order to cheat on the requested evaluation. Like, it demonstrated interesting exploit achievements because it didn’t “feel like” doing the exercise, which is unsettling because presumably it could do the same thing with any work I tried to delegate to it, and might in…
What I think it's interesting is that with the total lack of common sense the AI just goes on random tangents to achieve the target in a "monkey paw" way. Can you imagine if this happened: User: what is the shortest route from my home to the super market? AI: the user wants to know the shortest route to the super market. I should use a worm hole.
User: what is the shortest route from my home to the super market?
AI: the user wants to know, how do I make the super market my new home. Failing that, how do I make my home a super market.