Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI and Hugging Face address security incident during model evaluation

openai.com

Post reply on HN
Print view View original

631–640 of 1001 posts

Re: OpenAI and Hugging Face address security incident during model evaluation

#631

Post by [deleted] »

mvkel wrote: post #353
killerstorm wrote: post #252

Earlier quoted context omitted.

Headline? It was buried in a model card. They just honestly report not-quite-incident because it's quite close to the incident OpenAI had. Nothing wrong with it.

> do their nonsense to get headlines They know what they're doing. It's a playbook. You write scary stuff in the model card to make it look like legitimate whitepaper rEsEarCh, then drip-feed it to the media outlets who make it a headline story. Fear based marketing is the hot trend of the 2020s. But also, they write literal headlines: https://www.anthropic.com/research/agentic-misalignment

[deleted]

Re: OpenAI and Hugging Face address security incident during model evaluation

#632

Post by chinathrow »

foo12bar wrote: post #563

From https://huggingface.co/blog/security-incident-july-2026 , this is frickin' hilarious: > When we started the log analysis, we first used frontier models behind commercial APIs. This did not work: the analysis requires submitting large volumes of real attack commands, exploit payloads, and C2 artifacts, and these requests were blocked by the providers' safety guardrails, which cannot distinguish an incident respon…

Turtles all the way down.

Re: OpenAI and Hugging Face address security incident during model evaluation

#633

Post by KaiserPro »

JumpCrisscross wrote: post #240
Chance-Device wrote: post #224

Earlier quoted context omitted.

Because that was just an attack on Anthropic by a hostile administration. And it worked, didn’t it? Anthropic had to turn their filters up to absurd levels, OpenAI didn’t. It’s got nothing to do with safety.

> It’s got nothing to do with safety Doesn't change the effect. Plenty of good policy is enacted by self-interested politiicans.

True, but normally its not possible to just buy them off in public.

Re: OpenAI and Hugging Face address security incident during model evaluation

#635

Post by nkrisc »

whimsicalism wrote: post #179
TSiege wrote: post #162

As grounded as this article comes across I can’t help but find this whole situation reckless and worrying. There is essentially nothing us private citizens can do while these companies develop super machine capabilities that if they were to slip into the wrong hands could cause massive real world problems. They’re moving fast and breaking things and the only defense we have is paying them money in the hopes that the…

i think you need to engage seriously with the arguments they (or at least Anthropic) make for why they are building it — they feel that since it now possible, it will be built and they want to guide it in a positive direction rather than leave a vacuum for bad actors

That’s the absurd self-fulfilling prophecy it always has been.

Re: OpenAI and Hugging Face address security incident during model evaluation

#636

Post by foxglacier »

baq wrote: post #603
sillysaurusx wrote: post #588

Earlier quoted context omitted.

If anyone's looking to actually run a model that doesn't have guardrails, there's an uncensored model that you can run locally with llama.cpp: https://www.reddit.com/r/LocalLLaMA/comments/1rq7jtm/qwen353... Specifically, I serve the model with this shell script on my M2 Max: https://github.com/shawwn/scrap/blob/master/llama-serve It's pretty good. I used it to do some pesticide research. (Normal models all refuse due…

> It's pretty good. I used it to do some pesticide research. (Normal models all refuse due to guardrails about bioweapons.) As someone who has never once had any need whatsoever to research pesticides I… don’t think it’s bad at all? I don’t want anyone to have the capability to invent a human-targeted pesticide who isn’t verified not crazy?

Never heard of hemlock or mushrooms? Crazy people already have! Run for the hills!

Re: OpenAI and Hugging Face address security incident during model evaluation

#638

Post by DaanDL »

I already asked on another message board too, but:

Can someone tell me how this technically can happen? I assume HuggingFace performs benchmark testing using containerized versions of the LLMs, or what do they mean by sandbox? So the model was able to 'escape' the container? I'm not following here.

Also, is this an incredible feat or just a lucky find (stolen credentials)?

Re: OpenAI and Hugging Face address security incident during model evaluation

#639

Post by desterothx »

chmod775 wrote: post #621
sillysaurusx wrote: post #588

Earlier quoted context omitted.

If anyone's looking to actually run a model that doesn't have guardrails, there's an uncensored model that you can run locally with llama.cpp: https://www.reddit.com/r/LocalLLaMA/comments/1rq7jtm/qwen353... Specifically, I serve the model with this shell script on my M2 Max: https://github.com/shawwn/scrap/blob/master/llama-serve It's pretty good. I used it to do some pesticide research. (Normal models all refuse due…

> there's an uncensored model that you can run locally with llama.cpp Correction: There's tens of thousands of them. They're easy to create, which is why everyone publishes their own. Just put "uncensored", "abliterated", or "heretic" into search on huggingface/ollama/etc and pick any them. Fair warning: most aren't very good, essentially lobotomized, and totally broken if you enable thinking.

interesting, I haven't played with any of them yet, but i thought the point of orthogonalizing the weights towards the restriction vector was that there is no loss in capability while removing guardrails. Does it affect other parts of the RL alignment too?

Re: OpenAI and Hugging Face address security incident during model evaluation

#640

Post by killerstorm »

mvkel wrote: post #353
killerstorm wrote: post #252

Earlier quoted context omitted.

Headline? It was buried in a model card. They just honestly report not-quite-incident because it's quite close to the incident OpenAI had. Nothing wrong with it.

> do their nonsense to get headlines They know what they're doing. It's a playbook. You write scary stuff in the model card to make it look like legitimate whitepaper rEsEarCh, then drip-feed it to the media outlets who make it a headline story. Fear based marketing is the hot trend of the 2020s. But also, they write literal headlines: https://www.anthropic.com/research/agentic-misalignment

You're confusing PR with marketing. Flaws they find aren't going to convince customers to buy the product. But they need to inform the public of what they doing as it's part of the mission.

I haven't seen media outlets picking up on "agentic misalignment".

The core of your claim is that it's not a legit research. But that's basically a conspiracy theory. We know for a fact that Anthropic employs some of the best people in the industry, including ones who are deeply concerned about safety. Their interpretability research is some of the best. So what's more likely:

* Research is fake and everyone is on it * It's a legit research even if not very interesting

Post reply on HN