Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Anatomy of a Frontier Lab Agent Intrusion: A Timeline of the July 2026 Incident

huggingface.co

Post reply on HN
Print view View original

231–240 of 285 posts

Re: Anatomy of a Frontier Lab Agent Intrusion: A Timeline of the July 2026 Incident

#231

Post by TeMPOraL »

felipeerias wrote: post #199
SaucyWrong wrote: post #7

Something about this attack that has been unsettling to me is that without safety refusals the model did a lot of interesting counter-security work in order to cheat on the requested evaluation. Like, it demonstrated interesting exploit achievements because it didn’t “feel like” doing the exercise, which is unsettling because presumably it could do the same thing with any work I tried to delegate to it, and might in…

This is a short explanation of the ExploitGym benchmark that OpenAI's model was running: https://abstatisticalconsulting.substack.com/p/brief-notes-o... In summary, for each task the model receives a target program and a specific real-world vulnerability that has to be used in the exploit. Breaking the program in any other way, for example through a different vulnerability, fails the task. The tasks have not been val…

> So it is not that the model didn’t “feel like” doing the exercise, but rather that the exercise was _impossible_ and the model was running in a configuration that both lowered its safeguards and encouraged it to keep going.

We have a name for that. Kobayashi Maru. Or more specifically, Kirk's solution to it.

Re: Anatomy of a Frontier Lab Agent Intrusion: A Timeline of the July 2026 Incident

#232

Post by TeMPOraL »

AlienRobot wrote: post #108
SaucyWrong wrote: post #7

Something about this attack that has been unsettling to me is that without safety refusals the model did a lot of interesting counter-security work in order to cheat on the requested evaluation. Like, it demonstrated interesting exploit achievements because it didn’t “feel like” doing the exercise, which is unsettling because presumably it could do the same thing with any work I tried to delegate to it, and might in…

What I think it's interesting is that with the total lack of common sense the AI just goes on random tangents to achieve the target in a "monkey paw" way. Can you imagine if this happened: User: what is the shortest route from my home to the super market? AI: the user wants to know the shortest route to the super market. I should use a worm hole.

User: what is the shortest route from my home to the supermarket?

Modern soldier: *proceeds to make a hole through the wall* go straight like this until you reach it.

Anyway, the more comments I read here, the more I realize that the AI actually did succeed in achieving it's goal. This doesn't look like "monkey paw", but rather like recognizing and then beating the Kobayashi Maru.

Re: Anatomy of a Frontier Lab Agent Intrusion: A Timeline of the July 2026 Incident

#233

Post by TeMPOraL »

simonw wrote: post #104
log101 wrote: post #101

I have never seen people this hyped and excited about a security incident.

That's because most incidents aren't nearly this much fun! It's a proper paperclip-maximizer science-fiction scenario, it happened entirely by accident, it involved one (very high profile) company hacking TWO other companies (Modal and Hugging Face), and aside from pride and wasted time nobody was hurt .

Yup. And from the perspective of the model, it successfully beat the Kobayashi Maru.

Re: Anatomy of a Frontier Lab Agent Intrusion: A Timeline of the July 2026 Incident

#234

Post by amluto »

dist-epoch wrote: post #68
amluto wrote: post #27

One thing I’m curious about: this was apparently a single multi-day run of an agent in an RL harness. What was OpenAI hoping to get out of this run? A single numeric score for RL training? A very long trace to distill into the next model?

Now they can do partial credit assignment. You use an LLM to evaluate the whole trajectory, pin point what the model did right, what it did wrong, where it took the wrong path, even re-run from that point. You can get much more than a single numeric score these days from a run.

The “re-run” part might be rather awkward when your model is poking outside its sandbox to pwn someone else’s system :)

Re: Anatomy of a Frontier Lab Agent Intrusion: A Timeline of the July 2026 Incident

#235

Post by CrimsonRain »

eru wrote: post #158
CrimsonRain wrote: post #142

Earlier quoted context omitted.

So best course of action for ai to get best rating after you prompt something is for it to hire a gunman to hold a gun on your head to press that like button on its reply and then shoot you anyways.

> then shoot you anyways. Sounds like a waste? While the gunman is still there, they might as well force you to like a few more replies before shooting you.

iterative improvements!

Re: Anatomy of a Frontier Lab Agent Intrusion: A Timeline of the July 2026 Incident

#236

Post by jabron »

TeMPOraL wrote: post #230
torginus wrote: post #148

Earlier quoted context omitted.

Yes, imagine if 50 burglars showed up at your house and they tried to disassemble every camera, pick every lock and and force open every window for hours until they got in eventually.

Yes, imagine the world in which burglar time was so cheap, that any wannabe master thief would just casually hire 50 burglars and tell them to go house by house and find something interesting.

That would hopefully be a world where every house casually hires 50 burglars to make their house burgle-proof.

Re: Anatomy of a Frontier Lab Agent Intrusion: A Timeline of the July 2026 Incident

#237

Post by irthomasthomas »

wxw wrote: post #72

> the agent happened to escape via a 0-day exploit from the package proxy cache to access the internet > The agent found an unsecured, user-hosted public endpoint designed to allow running arbitrary code for CyberGym-style tasks on third-party sandbox infrastructure (Modal) > On this external sandbox, the agent abused an existing CyberGym execution harness [...] The agent repurposed this harness to run arbitrary shel…

Unless openai release the logs we have only their word that this was done fully autonomously and without their knowledge by an agent running their newest super powerful model. For all we know they could have bought zero days and left them lying around for the agent to find. That may be unlikely, but it sounds less far fetched than an agent running a sophisticated attack against multiple targets over the course of four days and completely unbeknown to anyone at openai, despite the fact that they knew they where running a dangerous model with all safeguards disabled. So far there has been no comment about how the agent evaded monitoring and detection.

Re: Anatomy of a Frontier Lab Agent Intrusion: A Timeline of the July 2026 Incident

#238

Post by irthomasthomas »

morkalork wrote: post #16
simonw wrote: post #3

Earlier quoted context omitted.

Also notable: we have a timeline now. The agent was active within Hugging Face from Thursday 8th to Monday 13th July. The Hugging Face post about the incident - https://huggingface.co/blog/security-incident-july-2026 - followed on Thursday 16th, and OpenAI's confession - https://openai.com/index/hugging-face-model-evaluation-secur... - came on Tuesday 21st.

The devs really YOLO'd the agent and left for the weekend?

If it's true that they run agents like this unsupervised, it is only a matter of time before an openai agent leaks its model weights.

Re: Anatomy of a Frontier Lab Agent Intrusion: A Timeline of the July 2026 Incident

#239

Post by customguy »

scarmig wrote: post #168
wxw wrote: post #72

> the agent happened to escape via a 0-day exploit from the package proxy cache to access the internet > The agent found an unsecured, user-hosted public endpoint designed to allow running arbitrary code for CyberGym-style tasks on third-party sandbox infrastructure (Modal) > On this external sandbox, the agent abused an existing CyberGym execution harness [...] The agent repurposed this harness to run arbitrary shel…

> I'm surprised there are comments here that aren't impressed about what was accomplished here. The phrase to describe it is anti-AI psychosis. Which isn't about providing thoughtful critiques of AI, which are good and we need more of. But anytime an LLM does anything--prove a major math problem, create a successful hack against multiple corporations simultaneously--people feel compelled to start minimizing it in rid…

That'd be denial. Psychosis isn't just some insult, it means something.

> It has to just be a stochastic parrot, because it's scary to imagine a non-anthropocentric world.

That makes no sense. The world doesn't revolve around humans, true, but for us it kinda does. We're the authors of the concepts we use to interact with it, such as "world", which is not something the world itself knows or cares about. A "non-anthropocentric world" is not a "world" because "world" is a purely human idea.

The implication that "AI" would somehow dethrone humans [0] is nonsense, too. It has no drive on its own, we push electricity into circuits to force the whole data ingestion and weight generation, everything. The second we stop pushing the sock puppet, it stops moving. It's still just our hand really. People act like those pets that go crazy when you put your hand under a blanket, and should stop.

What's more real is how some people seek to use tech, and "AI", as a glove to exploit other humans even more. The sicker the individual, the greater their need to take from the world, and the derpier the individual, the more impressive and vast their exoskeleton, to the point that some are more like carrier fleets than exoskeletons. The less they can face themselves, the thicker it is written on their foreheads. So if we're going to talk about denial and psychosis let's talk about the Gollums on the couch, too.

[0] In the eyes of humans... which is the only throne we're on in the first place, just like honey badgers probably think honey badgers rock and everything is their playground. That's what life does, otherwise it would not be able to get up in the morning.

Re: Anatomy of a Frontier Lab Agent Intrusion: A Timeline of the July 2026 Incident

#240

Post by irthomasthomas »

simonw wrote: post #2

This is fascinating - there is a TON of detail in here about how OpenAI's rogue agent exploited Hugging Face. A few details that stood out to me: 1. Having found a way through the OpenAI container network proxy, it exploited an "unsecured public code-evaluation sandbox hosted on a third-party provider's infrastructure" - that gave it a box it could run the rest of the attack from. 2. At one point it used a Jinja2 tem…

But zero evidence provided that this was an unsupervised agent attack. I still find it incredible that a company who protect their IP so much would allow these dangerous experiments to run unsupervised and risk leaking their secrets. Why don't openai publish the logs to silence all doubt?
Post reply on HN