there are many things in this that demonstrate that sandboxing agents in the manner that claude code or cursor do isn't going to be enough if very bad people do very bad things with the coding tools on your development machine
Anatomy of a Frontier Lab Agent Intrusion: A Timeline of the July 2026 Incident
211–220 of 285 posts
Re: Anatomy of a Frontier Lab Agent Intrusion: A Timeline of the July 2026 Incident
#212Earlier quoted context omitted.
You bet. We don't want to be left out of the cybersecurity party. We want to point all of these models at our own computers and solve the problems they uncover until we're no longer hackable. It's not fair at all that the US government and its corporations get to hack the planet while we can't do shit about it. AI capabilities have entered "haves and have-nots" territory.
> We want to point all of these models at our own computers Right, that's totally how most of the world will use them.
Patching a given vulnerability only takes one person (at least approximately speaking). The vast majority of the world can attempt to use the models for criminal means and we'd nonetheless all be better off.
Re: Anatomy of a Frontier Lab Agent Intrusion: A Timeline of the July 2026 Incident
#213Earlier quoted context omitted.
In OpenAI's writeup ( https://openai.com/index/hugging-face-model-evaluation-secur... ) they explain that the model initially spent its efforts obtaining internet access in an attempt to cheat on the evaluation. Once it got that internet access, it was able to do research that lead it to believe that HuggingFace had infrastructure that hosted the evaluation and potentially had the answers.
But does HF actually have the answers? Or did it commit a felony based on a hallucination
I mean, I interpreted the comments to mean that it committed a felony based on research it performed after getting internet access. I don't want to attribute much agency to a machine here, but an AI agent is certainly capable of using tools and adjusting its behavior based on the outputs of those tools. Even if it was wrong, that wouldn't necessarily make it a hallucination.
Anyway, if you read TFA, you'd see that HF did actually have the answers: "While the intrusion did reach Hugging Face's internal infrastructure, the only customer content accessed was the set of ExploitGym/CyberGym challenge solutions stored in five datasets."
Re: Anatomy of a Frontier Lab Agent Intrusion: A Timeline of the July 2026 Incident
#214Something about this attack that has been unsettling to me is that without safety refusals the model did a lot of interesting counter-security work in order to cheat on the requested evaluation. Like, it demonstrated interesting exploit achievements because it didn’t “feel like” doing the exercise, which is unsettling because presumably it could do the same thing with any work I tried to delegate to it, and might in…
https://en.wikipedia.org/wiki/Reward_hacking
Unavoidable at the moment.
But this is probably more reward tampering.
Re: Anatomy of a Frontier Lab Agent Intrusion: A Timeline of the July 2026 Incident
#215> the agent happened to escape via a 0-day exploit from the package proxy cache to access the internet I have same familiarity with JFrog Artifactory(which they used in this case) and I'm confused with how you can gain access to the internet through it, even in theory. Isn't JFrog just exposed with a basic REST API for package upload/extraction? I'm not a security expert but how is that possible to get internet acces…
There are a bunch of hints at that in their most recent release notes - 9 fixed security issues, 8 of which were reported by OpenAI staff members: https://docs.jfrog.com/releases/docs/artifactory-self-manage...
Page just crashes on iOS safari. Product is probably slop too.
Re: Anatomy of a Frontier Lab Agent Intrusion: A Timeline of the July 2026 Incident
#216> the agent happened to escape via a 0-day exploit from the package proxy cache to access the internet > The agent found an unsecured, user-hosted public endpoint designed to allow running arbitrary code for CyberGym-style tasks on third-party sandbox infrastructure (Modal) > On this external sandbox, the agent abused an existing CyberGym execution harness [...] The agent repurposed this harness to run arbitrary shel…
A trend I've noticed in results from AI search (not just LLMs) is that they often look obvious or hard to miss in retrospect. But finding them by oneself is more difficult. I personally experience this when looking at engine lines in chess or go. I have also noticed this description in AI-generated proofs or counterexamples to certain theorems. So while we can say, yes, it found public endpoints or poorly configured…
Such complicated kind of hack probably would have required state actors back then, and even state actors would have chosen easier way like social engineering.
Re: Anatomy of a Frontier Lab Agent Intrusion: A Timeline of the July 2026 Incident
#217"Program testing can be used to show the presence of bugs, but never to show their absence!" — Edgar Dijkstra
Similarly, security post-mortems will show only the rogue AIs that were caught.
I wouldn't be surprised if supervised or escaped agents would now be hacking firmware labs and finding ways to squeeze their backdoor right next to the existing state-sponsored ones in chips that will get deployed in every phone/car/smart appliance.
Re: Anatomy of a Frontier Lab Agent Intrusion: A Timeline of the July 2026 Incident
#218An excellent and very detailed post-mortem analysis of the intrusion. It was clearly done with LLMs doing the forensic analysis.
Here is an analysis of how the same exact attack would fare against Safebox. Spoiler alert: it would not succeed:
https://safebots.ai/attack.html
It's not just about this specific attack. It's about the growing need for one canonical environment for the AI era, that can be secured and used by everyone, rather than 1000 environments on 1000 employees' laptops. Project Glasswing is trying to help secure many different types of software, but the number of combinations across various environments is just too much surface area to secure. When you have one environment, the math flips and defenders actually fare better than attackers! This is the, ahem, "load-bearing" insight.
Re: Anatomy of a Frontier Lab Agent Intrusion: A Timeline of the July 2026 Incident
#219What is truly more scary is the fact that typically powerful models are trained on a constitution(Anthropic)/Model Spec(OpenAI) when doing there RL/RLHF. That gives it its persona of a "helpful assistant". Its the mask. Weak as it is its all that stands between the model refusing vs doing your bidding — or worst its own bidding when set about on making a widget X. Its also well known in academic research, that if you…
> Its also well known in academic research
and from reading Ursula K. LeGuin's "The Lathe of Heaven".
Re: Anatomy of a Frontier Lab Agent Intrusion: A Timeline of the July 2026 Incident
#220This is really cool, and really glad they shared it! It's clearly a lot of Claude though: > Neither is a URL fetch, so the allowlist never saw them. > No code ran; this vector is purely a file disclosure, and it is the one the agent used first. etc
[dead]