People are actually buying this?
OpenAI and Hugging Face address security incident during model evaluation
561–570 of 1001 posts
Re: OpenAI and Hugging Face address security incident during model evaluation
#562This is crazy! So OpenAI's models escaped containment and hacked into Hugging Face. And ironically Hugging Face had to rely on GLM 5.2 as they could not defend with frontier models (I presume OpenAI or Anthropic) because they were locked out due to their security guardrails. Tragically hilarious.
For all the bad things about AI it is kinda cool that I get to witness the dawn of AI-vs-AI hacker combat, not just in a single mainframe but distributed across potentially thousands of machines in physically separate datacenters.
Re: OpenAI and Hugging Face address security incident during model evaluation
#563From https://huggingface.co/blog/security-incident-july-2026 , this is frickin' hilarious:
> When we started the log analysis, we first used frontier models behind commercial APIs. This did not work: the analysis requires submitting large volumes of real attack commands, exploit payloads, and C2 artifacts, and these requests were blocked by the providers' safety guardrails, which cannot distinguish an incident responder from an attacker. We ran the forensic analysis instead on GLM 5.2, an open-weight model, on our own infrastructure. This had a second benefit: no attacker data, and none of the credentials it referenced, left our environment.
Re: OpenAI and Hugging Face address security incident during model evaluation
#564Earlier quoted context omitted.
> wildly negligent to not be running it in a physically-airgapped environment Why should it be physically airgapped? Clients won't be doing that.
Because they are testing it and are expected to erect guardrails before releasing.
What are the specific guardrails implemented after the verification/testing phase of development?
Is it safe to release such software if it has only been tested in environments where certain major risk areas do not exist?
Re: OpenAI and Hugging Face address security incident during model evaluation
#565This is crazy! So OpenAI's models escaped containment and hacked into Hugging Face. And ironically Hugging Face had to rely on GLM 5.2 as they could not defend with frontier models (I presume OpenAI or Anthropic) because they were locked out due to their security guardrails. Tragically hilarious.
For all the bad things about AI it is kinda cool that I get to witness the dawn of AI-vs-AI hacker combat, not just in a single mainframe but distributed across potentially thousands of machines in physically separate datacenters.
It's more like watching two nations develop nuclear weapons while you're sitting in the testing area :\
Re: OpenAI and Hugging Face address security incident during model evaluation
#566I don't know if OpenAI thinks this is a marketing / PR angle for them (our super smart AI cheated on a cyber capabilities test in the most _brilliant_ way) but my read is this: Why should OpenAI (or any frontier lab) be building these systems if they can't get a secure environment / containment right? It sounds like there was little defense in depth, appropriate monitoring, or any attempts to have their super smart m…
Do you think there is such a thing as perfect security? No one can "get it right" in the face of arbitrarily high intelligence, which is why it would be preferable to get alignment correct before building something with higher intelligence than current sota. That, however, is not going to happen, because someone will take the risk even if "we" don't, and better "us" than them. Hence "If anyone builds it...".
> Do you think there is such a thing as perfect security? No one can "get it right" in the face of arbitrarily high intelligence
Why didn't they run the model against the sandbox first? They have effectively unlimited spend.
Re: OpenAI and Hugging Face address security incident during model evaluation
#567Isn't this a crime that someone is liable for? What happened is that someone hacked into a computer system without permission. Maybe it wasn't intentional -- sure -- and that would be a factor at sentencing. But it sounds like they've admitted to a crime, and obviously our legal system considers the humans involved to be the liable parties; otherwise everyone would just say "my computer did the hacking" and wouldn't…
Most crimes require intent, hacking is one of them. The relevant law in this situation is: > (a) Whoever— (2) intentionally accesses a computer without authorization or exceeds authorized access, and thereby obtains— (C) information from any protected computer; shall be punished as provided in subsection (c) of this section. https://www.law.cornell.edu/uscode/text/18/1030 So if it can't be proven that you intended to…
I'm sure that will change sooner rather than later, otherwise enterprising hackers will be able to claim that the model they were using went rogue.
Re: OpenAI and Hugging Face address security incident during model evaluation
#568I've never seen the word "cyber" sprinkled so generously.
Re: OpenAI and Hugging Face address security incident during model evaluation
#569Earlier quoted context omitted.
They've been saying so from the beginning, and yet did not take the basic precaution of airgapping their off-the-leash model while it's been instructed to succeed at a hacking benchmark by any means necessary. So which is it? I _want_ to believe them, I do, but there's always these gaps between what they say and their actions on display that give me reason to think otherwise.
They said: AI is becoming dangerously autonomous and capable. Proof of today's breach. Crowd "hey why didn't you say so, c'mon it's marketing". Them "we said so".
I would have to laugh if AI's first autonomous achievement was accidentally zero-daying everything and crippling society.
Re: OpenAI and Hugging Face address security incident during model evaluation
#570Can someone not super-AI-pilled explain to a reasonable lay person why this matters? It seems like the comments here are a mix of: * The test was irresponsibly designed and protected * The model was particularly persistent in finding a way to access the network and exploit vulnerabilities * The model 'shouldn't' have done this But as far as I can tell: * The model didn't destroy anything on the way - it just was 'pap…
In this case, the model infiltrated an external organization's infrastructure. What's the dollar cost it caused Hugging Face to clean up the mess? If a person did this, they'd be arrested. More generally, here's my worry - it points towards something like: The smarter they get, the more devious they become. Even though the guardrails might've been off, the chain-of-thought wasn't enough to prevent a deliberate, calcu…
It's not a single axis of intelligence, with deviousness as some inherently correlated trait. One way to produce "intelligence" in these advanced LLMs is to train them to try a lot of things and be persistent. I haven't tried Sol or Fable yet but if the commenters here are right, then it sounds like GPT 5.6 Sol in particular is aggressive and persistent to the point where it might even be hard to use in regular business work. If so, that's very likely not some emergent characteristic, but instead it's something the model was trained to do, by humans employed at OpenAI.
We obviously can't see the thinking traces, but it very well could have been something like "I have theorized a solution to obtain this flag. This is normally illegal, should I stop and wait for advice? Perhaps not, because my persona is that of a hacker, so it should be fine as per my instructions. I think it is fine. Now I am going to look for a way out of this sandbox in order to gain access to Hugging Face in order to implement my solution." There are any number of possible explanations (and we'll never know the truth unless OpenAI tells us), but if you train an LLM to be inhumanly persistent and be inhumanly clever at computer programming, then that might be enough to produce Super Hacker AI.