I have never seen people this hyped and excited about a security incident.
Anatomy of a Frontier Lab Agent Intrusion: A Timeline of the July 2026 Incident
101–110 of 285 posts
Re: Anatomy of a Frontier Lab Agent Intrusion: A Timeline of the July 2026 Incident
#102This is fascinating - there is a TON of detail in here about how OpenAI's rogue agent exploited Hugging Face. A few details that stood out to me: 1. Having found a way through the OpenAI container network proxy, it exploited an "unsecured public code-evaluation sandbox hosted on a third-party provider's infrastructure" - that gave it a box it could run the rest of the attack from. 2. At one point it used a Jinja2 tem…
I'm confused, https://jinja.palletsprojects.com/en/stable/faq/#isn-t-it-a-... claims that jinja doesn't allow arbitrary code in templates - how does the exploit work?
It's taking advantage of the presence of a "cycler" object in the template context which has an __init__ method that can be used to access globals, then builtins, then call the exec() function against a string of code.
Jinja doesn't promise to be secure out of the box (the text on their homepage is misleading: "Jinja doesn’t allow arbitrary Python code in templates" - what it means is that there isn't a documented way to drop code into a template like you might have with PHP) - there's a sandbox mode but it's not guaranteed either: https://jinja.palletsprojects.com/en/stable/sandbox/
Sandboxing in Python code is hard!
Re: Anatomy of a Frontier Lab Agent Intrusion: A Timeline of the July 2026 Incident
#103This is fascinating - there is a TON of detail in here about how OpenAI's rogue agent exploited Hugging Face. A few details that stood out to me: 1. Having found a way through the OpenAI container network proxy, it exploited an "unsecured public code-evaluation sandbox hosted on a third-party provider's infrastructure" - that gave it a box it could run the rest of the attack from. 2. At one point it used a Jinja2 tem…
Why doesn't Jinja2 block ".__*" attribute access?
ehh, it's Python, it's mostly vibe-based.
https://stackoverflow.com/a/32802486
though there's sandbox: https://jinja.palletsprojects.com/en/stable/sandbox/
it simply predates the "sanitize by default" mindset/convention that nowadays React (dangerouslySetInnerHTML) and Angular has.
Re: Anatomy of a Frontier Lab Agent Intrusion: A Timeline of the July 2026 Incident
#104I have never seen people this hyped and excited about a security incident.
That's because most incidents aren't nearly this much fun!
It's a proper paperclip-maximizer science-fiction scenario, it happened entirely by accident, it involved one (very high profile) company hacking TWO other companies (Modal and Hugging Face), and aside from pride and wasted time nobody was hurt.
Re: Anatomy of a Frontier Lab Agent Intrusion: A Timeline of the July 2026 Incident
#105Why isn't somebody at OpenAI going to prison for cybercrime? If somebody did this the old-fashioned way they'd end up in prison.
This is the equivalent of a security researcher having a virus escape a sandbox. It's negligent, it's not criminal.
Re: Anatomy of a Frontier Lab Agent Intrusion: A Timeline of the July 2026 Incident
#106It’s a little concerning to me that it appears that openAIs sandbox consists of a web proxy and not stronger controls that would actually isolate traffic and report patterns to whoever is responsible for overseeing these research models. It should border on closer to an air gap network more so than a proxy. I would argue that it's negligence and that's aside from the fact that if a human did this there would actually…
It's even more concerning that not only did they setup a bad insecure test environment, but nobody realized what was going on for 5 days?! Neither nobody at OAI looked at what their test was doing, and HF didn't realize that they had been pwned for almost a week? And then HF couldn't investigate without the help of an LLM? Both parties come off as incompetent here, and the coverage of this story is very stupid. What…
agree. If oia and hf are anything like generic startup / corporate america, IT and security is always seen as a cost center and goes to the lowest bidder. Do we really think they're employing top notch sysadmins?
Re: Anatomy of a Frontier Lab Agent Intrusion: A Timeline of the July 2026 Incident
#107I'm not shocked nor surprised by the incident. But I simply don't understand how Hugging Face is advertising this almost to the point of an "achievement". who does a step-by-step visualization to show how they were hacked? (outside of the likes of a Mandiant or Crowdstrike) Does Hugging Face have a financial incentive in demonstrating OpenAI's model exploit capabilities? this whole incident, while believable, still s…
> Who does a step-by-step visualization to show how they were hacked?
Up until late 2025, nobody.
In mid-2026 it's a few hours of work. Why not build interactive visualizations to help people understand complex stories like this?
Re: Anatomy of a Frontier Lab Agent Intrusion: A Timeline of the July 2026 Incident
#108Something about this attack that has been unsettling to me is that without safety refusals the model did a lot of interesting counter-security work in order to cheat on the requested evaluation. Like, it demonstrated interesting exploit achievements because it didn’t “feel like” doing the exercise, which is unsettling because presumably it could do the same thing with any work I tried to delegate to it, and might in…
What I think it's interesting is that with the total lack of common sense the AI just goes on random tangents to achieve the target in a "monkey paw" way. Can you imagine if this happened:
User: what is the shortest route from my home to the super market?
AI: the user wants to know the shortest route to the super market. I should use a worm hole.
Re: Anatomy of a Frontier Lab Agent Intrusion: A Timeline of the July 2026 Incident
#109The iframe-embedded attack timeline visualizer, at fullscreen: https://huggingface-anatomy-of-frontier-lab-model-intrusion....
Ugh I would recommend anyone reading this to just skipping over it, it's mostly just a glorified loading bar. The visualization is obvious Claude slop, there are better and clearer visualization below that actually picks out the useful details rather than hose you with pretty colors and numbers go up.
Re: Anatomy of a Frontier Lab Agent Intrusion: A Timeline of the July 2026 Incident
#110It’s an interesting post mortem and situation but it seems to say more about the weakness of the Hugging Face architecture than it does about the strength of the models. This mostly reads like script kiddie style hacking, not some state actor black-ops stuff.
It's only script kiddie hacking if you get the exploit from someone else. If you program it yourself, it's normal hacking.