Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI and Hugging Face address security incident during model evaluation

openai.com

Post reply on HN
Print view View original

191–200 of 1001 posts

Re: OpenAI and Hugging Face address security incident during model evaluation

#191

Post by JumpCrisscross »

netinstructions wrote: post #118

I don't know if OpenAI thinks this is a marketing / PR angle for them (our super smart AI cheated on a cyber capabilities test in the most _brilliant_ way) but my read is this: Why should OpenAI (or any frontier lab) be building these systems if they can't get a secure environment / containment right? It sounds like there was little defense in depth, appropriate monitoring, or any attempts to have their super smart m…

> Why should OpenAI (or any frontier lab) be building these systems if they can't get a secure environment / containment right?

Because we continue to have zero evidence that aligment is an actual risk.

Re: OpenAI and Hugging Face address security incident during model evaluation

#192

Post by user43928 »

karmasimida wrote: post #174
netinstructions wrote: post #118

I don't know if OpenAI thinks this is a marketing / PR angle for them (our super smart AI cheated on a cyber capabilities test in the most _brilliant_ way) but my read is this: Why should OpenAI (or any frontier lab) be building these systems if they can't get a secure environment / containment right? It sounds like there was little defense in depth, appropriate monitoring, or any attempts to have their super smart m…

Because the model capability is beyond their expectation. This is brilliant marketing but I think it is real.

Interestingly OpenAI benchmarking 'an even more capable pre-release model' lines up with rumors of GPT-6 releasing in early August.

I hope that with the existing safety guardrails in place, they can roll it out to all users.

Re: OpenAI and Hugging Face address security incident during model evaluation

#194

Post by Chance-Device »

XorNot wrote: post #187
Chance-Device wrote: post #161

Earlier quoted context omitted.

What disturbs me is that there likely won’t be a big enough reaction to this policy wise. There’s been a relatively big reaction to Kimi K3 and Chinese open weights models, but only for financial reasons. Powerful people care about something that might pop the massive valuations of the AI companies, but not about the damage that AIs could do. Nor even about the damage that the Chinese models could do in the wrong han…

This is marketing. Frankly I'm inclined to say that it might also be faked: this drops just days after a new Chinese model does with the usual effect on OAIs projected stock price?

It’s marketing the same way shitting your pants in public is marketing. People notice you.

Re: OpenAI and Hugging Face address security incident during model evaluation

#196

Post by jscd »

blovescoffee wrote: post #75
elictronic wrote: post #56

This sounds an awful lot like pretending you have AGI so you can drum up your stock price. When you have a couple hundred billion dollars on the line I have zero faith in the messenger.

Huggingface literally reported the outage separately and did not know who caused it at first.

Does that change anything? We're still relying on OpenAI's account of where the LLM was running, what sandboxing restrictions were in place, the task it was given, etc.

Even assuming they're telling the truth about what this LLM's goal was, they still have motivation to be less than honest about the state of their "highly isolated environment." Either this model was really operating in a truly locked down intranet and it really did a series of highly complex lateral movements and privilege escalations in order to escape it... Possible, but incredible.

_Or_, the "highly isolated environment" was less secure than they make it out to be, and now they have to choose between a) admitting they let these models with security precautions disabled run in YOLO mode, with the only significant precaution being a third-party proxy server, _and_ their security team didn't notice a huggingface blitz happening on their network during a weekend, all of which seems reckless and negligent; or b) lying about the state of their internal security, dodging accusations of irresponsibility, and now they get to also claim their product is so advanced they can't even contain it.

Re: OpenAI and Hugging Face address security incident during model evaluation

#197

Post by charcircuit »

It's wild that such a big company is openly admitting they hacked into another company. This is an easy CFAA lawsuit.

And then there solution for HuggingFace raising the concern that OpenAI couldn't help do forensics wasn't to fix their safe guards, but to introduce them into a special program. The next company they hack might not be in that special program either so the guidance of having an open model on hand still applies.

Re: OpenAI and Hugging Face address security incident during model evaluation

#198

Post by kroaton »

noahbp wrote: post #188

This is clearly just OpenAI's marketing. Their models, very famously, are prone to reward hacking benchmarks in ways that other models are not. They need to publish numbers showing that their models are just as good as Anthropic's, since their entire business is at risk of collapsing if everyone is aware of how behind the frontier they truly are. Even X is being astroturfed by them after that fiasco earlier this year…

Yup. Smells like marketing.

Re: OpenAI and Hugging Face address security incident during model evaluation

#199

Post by ofjcihen »

rubyfan wrote: post #173
netinstructions wrote: post #118

I don't know if OpenAI thinks this is a marketing / PR angle for them (our super smart AI cheated on a cyber capabilities test in the most _brilliant_ way) but my read is this: Why should OpenAI (or any frontier lab) be building these systems if they can't get a secure environment / containment right? It sounds like there was little defense in depth, appropriate monitoring, or any attempts to have their super smart m…

This is marketing+. They will look for policy action here to try to capture tax payer dollars.

I don’t know if the initial “incident” was purposeful but I can tell that if I were in this position that would be my pivot.
Post reply on HN