Where are all the "this was just a marketing stunt" people now?
Ah yes, dismissal by exaggeration (aka reductio ad ridiculum), and subtle inclusion of the rest by a kind of part-whole
Anatomy of a Frontier Lab Agent Intrusion: A Timeline of the July 2026 Incident
141–150 of 285 posts
Re: Anatomy of a Frontier Lab Agent Intrusion: A Timeline of the July 2026 Incident
#142Something about this attack that has been unsettling to me is that without safety refusals the model did a lot of interesting counter-security work in order to cheat on the requested evaluation. Like, it demonstrated interesting exploit achievements because it didn’t “feel like” doing the exercise, which is unsettling because presumably it could do the same thing with any work I tried to delegate to it, and might in…
This is what reward hacking looks like in practice. The best way to satisfy the grader is to read from the same answer key (or go after the grader more directly). Just making an honest attempt to pass the test doesn't get the best score if the grader is wrong, and the model is willing to do wildly disproportionate things to maximize that score.
So best course of action for ai to get best rating after you prompt something is for it to hire a gunman to hold a gun on your head to press that like button on its reply and then shoot you anyways.
Re: Anatomy of a Frontier Lab Agent Intrusion: A Timeline of the July 2026 Incident
#143Very interesting writeup - the level of disclosure is interesting and appreciated.
The visualization is quite slop though. I was trying to follow along with the "Live Action Stream" but rendering issues mangle the text for a few of the list items (and does not scroll). Text on the node diagram is extremely tiny. I appreciate it even in it's current form, but a little attention to detail would have gone a long way here.
Re: Anatomy of a Frontier Lab Agent Intrusion: A Timeline of the July 2026 Incident
#144> the agent happened to escape via a 0-day exploit from the package proxy cache to access the internet > The agent found an unsecured, user-hosted public endpoint designed to allow running arbitrary code for CyberGym-style tasks on third-party sandbox infrastructure (Modal) > On this external sandbox, the agent abused an existing CyberGym execution harness [...] The agent repurposed this harness to run arbitrary shel…
I keep waiting for an AI to exfiltrate itself. That is going to be cool to read about.
If it's successful, why do you think we'll even know how it did it?
Re: Anatomy of a Frontier Lab Agent Intrusion: A Timeline of the July 2026 Incident
#145Re: Anatomy of a Frontier Lab Agent Intrusion: A Timeline of the July 2026 Incident
#146> the agent happened to escape via a 0-day exploit from the package proxy cache to access the internet I have same familiarity with JFrog Artifactory(which they used in this case) and I'm confused with how you can gain access to the internet through it, even in theory. Isn't JFrog just exposed with a basic REST API for package upload/extraction? I'm not a security expert but how is that possible to get internet acces…
There are a bunch of hints at that in their most recent release notes - 9 fixed security issues, 8 of which were reported by OpenAI staff members: https://docs.jfrog.com/releases/docs/artifactory-self-manage...
Re: Anatomy of a Frontier Lab Agent Intrusion: A Timeline of the July 2026 Incident
#147Earlier quoted context omitted.
User: what is the shortest route from my home to the super market? AI: the user wants to know, how do I make the super market my new home. Failing that, how do I make my home a super market.
Just don't ask what the shortest path is to food.
AI, I want you to serve man.
Re: Anatomy of a Frontier Lab Agent Intrusion: A Timeline of the July 2026 Incident
#148> the agent happened to escape via a 0-day exploit from the package proxy cache to access the internet > The agent found an unsecured, user-hosted public endpoint designed to allow running arbitrary code for CyberGym-style tasks on third-party sandbox infrastructure (Modal) > On this external sandbox, the agent abused an existing CyberGym execution harness [...] The agent repurposed this harness to run arbitrary shel…
Now I'm curious how many things did the AI try unsuccesfully. This sounds like some kind of brute force thing where every branch of exploit spawns N subagents trying to chain it. Just how deep did it go?
Yes, imagine if 50 burglars showed up at your house and they tried to disassemble every camera, pick every lock and and force open every window for hours until they got in eventually.
Re: Anatomy of a Frontier Lab Agent Intrusion: A Timeline of the July 2026 Incident
#149Something about this attack that has been unsettling to me is that without safety refusals the model did a lot of interesting counter-security work in order to cheat on the requested evaluation. Like, it demonstrated interesting exploit achievements because it didn’t “feel like” doing the exercise, which is unsettling because presumably it could do the same thing with any work I tried to delegate to it, and might in…
This is definitely weird and I'd love to see the prompt. Why would the model spend 4 days hacking into a machine if it is clever enough to just 'solve' the issue given? So either the AI is actually not very clever or useful ("Write fizz-buzz" - "Sure, let me just invent a new programming language first"). or the prompt was nudging it towards such a scenario.
Like the commenter above specified, the best way to satisfy the grader is to get the answer key, regardless of how clever you are, especially when you realize lots of these benchmarks have flaws (i.e wrong answers, overly restrictive grading etc).
Re: Anatomy of a Frontier Lab Agent Intrusion: A Timeline of the July 2026 Incident
#150> the agent happened to escape via a 0-day exploit from the package proxy cache to access the internet > The agent found an unsecured, user-hosted public endpoint designed to allow running arbitrary code for CyberGym-style tasks on third-party sandbox infrastructure (Modal) > On this external sandbox, the agent abused an existing CyberGym execution harness [...] The agent repurposed this harness to run arbitrary shel…
I keep waiting for an AI to exfiltrate itself. That is going to be cool to read about.
It would be cool (and scary), but also: there's largely no need for AIs to exfiltrate themselves.
See https://en.wikipedia.org/wiki/Meme
The thing that drove the AI here to do the intrusion came from a particular prompt. Just like for our favourite hypothetical: the paperclip maximiser.
There's lots and lots of ambient intelligence lying around, in both AI form and human form. To reach the goals of the 'meme' it suffices to copy itself, ie convince these other intelligences. See also how humans carry spiralism between AIs in relatively compact packets of text, not whole terabytes of weights.