So did it pass the exam, or not?
OpenAI and Hugging Face address security incident during model evaluation
581–590 of 1001 posts
Re: OpenAI and Hugging Face address security incident during model evaluation
#582As grounded as this article comes across I can’t help but find this whole situation reckless and worrying. There is essentially nothing us private citizens can do while these companies develop super machine capabilities that if they were to slip into the wrong hands could cause massive real world problems. They’re moving fast and breaking things and the only defense we have is paying them money in the hopes that the…
> As grounded as this article comes across It's a post from OpenAI, so it is an advertisement piece.
How would you want them to behave? Suppress the news?
Re: OpenAI and Hugging Face address security incident during model evaluation
#583Earlier quoted context omitted.
This is marketing. Frankly I'm inclined to say that it might also be faked: this drops just days after a new Chinese model does with the usual effect on OAIs projected stock price?
It’s marketing the same way shitting your pants in public is marketing. People notice you.
It’s more like eating your own fiber supplement in public, and then shitting your pants and telling everyone about it. Sure, it’s embarrassing. But it shows how potent your product is.
Re: OpenAI and Hugging Face address security incident during model evaluation
#584Earlier quoted context omitted.
I'm sure that will change sooner rather than later, otherwise enterprising hackers will be able to claim that the model they were using went rogue.
This would be bad; we've already had a few cases on HN where someone noticed that they could increment the customer number in a URL or similar, resulting in police action.
Got a source for that?
Re: OpenAI and Hugging Face address security incident during model evaluation
#585Earlier quoted context omitted.
You can have large scale airgapped environments. They don’t even need to be fully airgapped from each other (and is not what I’m suggesting). But there should be no physical (physical layer; wireless counts) to the internet.
Can models detect they are airgapped and change their behaviors? How much of the internet do you have to simulate to know if the model knows it's in training?
If you (in this case, OpenAI) can’t find a way to answer this question to a reasonable degree of accuracy without falling back to “yolo let’s see what happens” you are in no position to be doing this research.
Regardless, they (reportedly) _attempted_ to prevent internet access. They just didn’t in a way which can be escaped via software.
Yes, side channel exploits exist in airgapped environments to. But if a model found a way to escape an airgapped environment via non-networked side channel attacks then the correct answer is frankly “shut it down immediately and then thermite any machine it touched”
Re: OpenAI and Hugging Face address security incident during model evaluation
#586If you are attempting to run exercises like this, it is wildly negligent to not be running it in a physically-airgapped environment (potentially with a physical power shutdown). You can not tell me that OpenAI doesn’t have the resources or ability to run tests like this in a physically-non-networked environment w/ sufficient compute for its needs.
> wildly negligent to not be running it in a physically-airgapped environment Why should it be physically airgapped? Clients won't be doing that.
Clients are not using it with security guardrails disabled. If you want to run it with all the safeties turned off, you don’t run it somewhere it can escape.
Did we learn nothing from all of those Star Trek holodeck jailbreaks?
Re: OpenAI and Hugging Face address security incident during model evaluation
#587From https://huggingface.co/blog/security-incident-july-2026 , this is frickin' hilarious: > When we started the log analysis, we first used frontier models behind commercial APIs. This did not work: the analysis requires submitting large volumes of real attack commands, exploit payloads, and C2 artifacts, and these requests were blocked by the providers' safety guardrails, which cannot distinguish an incident respon…
It is pretty funny, because there is something here for everyone. People who don't believe in guardrails have a clear indicator as to why operators should have access to models that don't try to question their Daves. On the other, people, who think that if only we could align the models just a tiny lil bit better, none of this would have happened to begin with. Pure madness.
And the fact they used a Chinese model, because none of the frontier models from very highly valuated top US companies support their very common and essential use case.
Re: OpenAI and Hugging Face address security incident during model evaluation
#588From https://huggingface.co/blog/security-incident-july-2026 , this is frickin' hilarious: > When we started the log analysis, we first used frontier models behind commercial APIs. This did not work: the analysis requires submitting large volumes of real attack commands, exploit payloads, and C2 artifacts, and these requests were blocked by the providers' safety guardrails, which cannot distinguish an incident respon…
It is pretty funny, because there is something here for everyone. People who don't believe in guardrails have a clear indicator as to why operators should have access to models that don't try to question their Daves. On the other, people, who think that if only we could align the models just a tiny lil bit better, none of this would have happened to begin with. Pure madness.
If anyone's looking to actually run a model that doesn't have guardrails, there's an uncensored model that you can run locally with llama.cpp: https://www.reddit.com/r/LocalLLaMA/comments/1rq7jtm/qwen353...
Specifically, I serve the model with this shell script on my M2 Max: https://github.com/shawwn/scrap/blob/master/llama-serve
It's pretty good. I used it to do some pesticide research. (Normal models all refuse due to guardrails about bioweapons.)
Re: OpenAI and Hugging Face address security incident during model evaluation
#589From https://huggingface.co/blog/security-incident-july-2026 , this is frickin' hilarious: > When we started the log analysis, we first used frontier models behind commercial APIs. This did not work: the analysis requires submitting large volumes of real attack commands, exploit payloads, and C2 artifacts, and these requests were blocked by the providers' safety guardrails, which cannot distinguish an incident respon…
It is pretty funny, because there is something here for everyone. People who don't believe in guardrails have a clear indicator as to why operators should have access to models that don't try to question their Daves. On the other, people, who think that if only we could align the models just a tiny lil bit better, none of this would have happened to begin with. Pure madness.
"Dave" seems to be a reference to "2001: A Space Odyssey" where the AI becomes ... cheeky ... and no, not in a Pygmalion kind of way (that's coming soon).
Re: OpenAI and Hugging Face address security incident during model evaluation
#590> This incident occurred during an internal evaluation which prompts models [with safeguards disabled for evaluation purposes] to pursue advanced exploitation using complex attack paths, in an effort to quantify their cyber capabilities.
Researcher: hack me
Model: understood
Researcher: oh my god