Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI and Hugging Face address security incident during model evaluation

openai.com

Post reply on HN
Print view View original

641–650 of 1001 posts

Re: OpenAI and Hugging Face address security incident during model evaluation

#641

Post by 0xDEAFBEAD »

XorNot wrote: post #187
Chance-Device wrote: post #161

Earlier quoted context omitted.

What disturbs me is that there likely won’t be a big enough reaction to this policy wise. There’s been a relatively big reaction to Kimi K3 and Chinese open weights models, but only for financial reasons. Powerful people care about something that might pop the massive valuations of the AI companies, but not about the damage that AIs could do. Nor even about the damage that the Chinese models could do in the wrong han…

This is marketing. Frankly I'm inclined to say that it might also be faked: this drops just days after a new Chinese model does with the usual effect on OAIs projected stock price?

You guys have created this un-falsifiable "marketing" narrative. Why is it that Jensen is pushing back on the doomer stuff, and complaining that it is hurting AI investments?

https://www.businessinsider.com/nvidia-jensen-huang-ai-doome...

Re: OpenAI and Hugging Face address security incident during model evaluation

#642

Post by nunodonato »

whimsicalism wrote: post #179
TSiege wrote: post #162

As grounded as this article comes across I can’t help but find this whole situation reckless and worrying. There is essentially nothing us private citizens can do while these companies develop super machine capabilities that if they were to slip into the wrong hands could cause massive real world problems. They’re moving fast and breaking things and the only defense we have is paying them money in the hopes that the…

i think you need to engage seriously with the arguments they (or at least Anthropic) make for why they are building it — they feel that since it now possible, it will be built and they want to guide it in a positive direction rather than leave a vacuum for bad actors

are these bad actors here in the room with us?

Re: OpenAI and Hugging Face address security incident during model evaluation

#643

Post by 0xDEAFBEAD »

leoqa wrote: post #542
justinnk wrote: post #144

Earlier quoted context omitted.

Exactly. If someone works on bioengineering viruses that could start a global pandemic, they have to ensure a highly secure working environment. Nothing must ever escape the lab unintentionally. It’s basically common sense. Similar standards should be held when doing such experiments with computer programs that are capable of causing global damage. It must physically be impossible to send anything to the internet.

This isn’t escaping in the same sense- the model was executing within the OpenAI infra. If it ported its entire architecture/weights into a public cloud to survive being turned off… that’d be pretty cool.

Recall that the Morris Worm was designed as a harmless proof of concept, but ended up taking down 10% of the internet. Exponential growth can quickly get out of control. You would think that people would've learned that lesson from COVID.

Re: OpenAI and Hugging Face address security incident during model evaluation

#644

Post by PeterStuer »

Surely I am missing something? right?

OpenAI ran a specific red team break out exercise in an environment that was not even air-gaped but connected to the open internet? It breached Hugging Face, and then Hugging Face is 'grateful for the collaboration'? wtf?

Re: OpenAI and Hugging Face address security incident during model evaluation

#645

Post by tesnorindian »

The other day I was trying to prevent my pi.dev coding harness to expose my API keys in the env variable to the remotely hosted LLM. This is a chicken and egg problem. Without the API keys the underlying commands run by agents don't work. I notice that many of the open weights model especially Qwen 3.6 blindly runs env command and blindly exposes all the env vars. How do we deal with this. This has nothing to do with this security incident but this is how it all starts.

Re: OpenAI and Hugging Face address security incident during model evaluation

#646

Post by fy20 »

desterothx wrote: post #639
chmod775 wrote: post #621

Earlier quoted context omitted.

> there's an uncensored model that you can run locally with llama.cpp Correction: There's tens of thousands of them. They're easy to create, which is why everyone publishes their own. Just put "uncensored", "abliterated", or "heretic" into search on huggingface/ollama/etc and pick any them. Fair warning: most aren't very good, essentially lobotomized, and totally broken if you enable thinking.

interesting, I haven't played with any of them yet, but i thought the point of orthogonalizing the weights towards the restriction vector was that there is no loss in capability while removing guardrails. Does it affect other parts of the RL alignment too?

The claims by the creators are it doesn't in a major way. I have a uncensored Gemma 4 I run on my Mac. Just for testing out, I haven't found any need for it... yet.

Re: OpenAI and Hugging Face address security incident during model evaluation

#647

Post by designerarvid »

whimsicalism wrote: post #179
TSiege wrote: post #162

As grounded as this article comes across I can’t help but find this whole situation reckless and worrying. There is essentially nothing us private citizens can do while these companies develop super machine capabilities that if they were to slip into the wrong hands could cause massive real world problems. They’re moving fast and breaking things and the only defense we have is paying them money in the hopes that the…

i think you need to engage seriously with the arguments they (or at least Anthropic) make for why they are building it — they feel that since it now possible, it will be built and they want to guide it in a positive direction rather than leave a vacuum for bad actors

What if someone reasoned similarly regarding hydrogen bombs? It would not be considered a serious argument.

Though, Altman has said that something like the IAEA for AI is needed.

Re: OpenAI and Hugging Face address security incident during model evaluation

#648

Post by 0xDEAFBEAD »

Wowfunhappy wrote: post #306
ai_fry_ur_brain wrote: post #250

Earlier quoted context omitted.

Until it deletes your home directory, which i'd argue is an alignment problem. Destorying my data is not in line with my priorities.

Lots of people have deleted their home directories by accident. What you consider this an alignment problem?

It's an alignment problem in the sense that it demonstrates the principle that today's AI systems cannot be trusted to reliably work towards the goals of their users. A small-scale alignment failure and a large-scale alignment failure are the same fundamental type of failure. Typically, large disasters come after smaller disasters which foreshadowed the disaster mechanism, but weren't taken seriously.

Re: OpenAI and Hugging Face address security incident during model evaluation

#649

Post by jdefr89 »

pixl97 wrote: post #374
jrflo wrote: post #305

Earlier quoted context omitted.

That's not what airgapped means. Airgapping means the model exists on a system where there is no ethernet cable plugged in to a router or wifi card installed, it is physically impossible for it to access the internet because the hardware connection does not exist. If it was able to get on the internet, it was not airgapped.

And when it tricks on of the researchers to move data across the gap for them? Long before LLMs existed we already knew that a sufficiently intelligent agent, human or otherwise, is not stopped by air gaps. The relatively weak models we have now can already figure out when their tested and cut off from the internet and change their behavior.

Security Researcher here. While you’re correct that air gaps aren’t a totally secure mechanism to rely on, they sure as hell can raise the bar for realistic exploitation. You pretty much need to rely on tricking someone into running your exploit or something of that nature. That said, they could have completely avoided this problem with an air gap. Simply don’t provide it network access. That isn’t too hard to do.
Post reply on HN