Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI and Hugging Face address security incident during model evaluation

openai.com

Post reply on HN
Print view View original

221–230 of 1001 posts

Re: OpenAI and Hugging Face address security incident during model evaluation

#221

Post by scoring1774 »

This is the first one of these announcements that has me actually scared of what comes next. Obviously these models have gotten smarter but this strikes me as the first time I've seen a model have a "paperclip factory" moment and perform non-trivial tasks to accomplish a clearly misaligned secondary goal.

It's remarkable that building a society based around having to do something so you can go do your hobbies at home after work has built tools like this. I still just want to play music so I hope we can control these enough to make that possible without detonating what I love.

Re: OpenAI and Hugging Face address security incident during model evaluation

#222

Post by cloudie78 »

Until they disclose the actual technical details of their “highly sophisticated sandbox environment” or whatever the hell the wording they used is - they can kindly do us all a favour and fuck off.

It’s over, there’s no moat, only the gullible idiots remain.

Re: OpenAI and Hugging Face address security incident during model evaluation

#224

Post by Chance-Device »

urams wrote: post #218
Chance-Device wrote: post #161

Earlier quoted context omitted.

What disturbs me is that there likely won’t be a big enough reaction to this policy wise. There’s been a relatively big reaction to Kimi K3 and Chinese open weights models, but only for financial reasons. Powerful people care about something that might pop the massive valuations of the AI companies, but not about the damage that AIs could do. Nor even about the damage that the Chinese models could do in the wrong han…

> What disturbs me is that there likely won’t be a big enough reaction to this policy wise. Anthropic was blocked from releasing Fable without any such level of incident. OAI was also briefly blocked from releasing 5.6. Why do you think there is no policy appetite?

Because that was just an attack on Anthropic by a hostile administration. And it worked, didn’t it? Anthropic had to turn their filters up to absurd levels, OpenAI didn’t. It’s got nothing to do with safety.

Re: OpenAI and Hugging Face address security incident during model evaluation

#225

Post by _pdp_ »

arisAlexis wrote: post #136
_pdp_ wrote: post #94

Earlier quoted context omitted.

I am not saying it is marketing but typically when there is a data breach you may hear from the CISO but most of the time is is vague PR response. In this case I get loud signals from both HG and OpenAI leadership without much information exactly what the attack was about just that GPT x.x was involved. It is unusual all I am trying to say.

It's incredible how people miss the forest for the trees thinking constantly that Sam and Dario are marketing gurus when they are literally trying to contain nuclear material. Not sure what has to happen for this thinking to stop maybe a huge accident and the. Aha maybe they had a point

I don't think there is any dispute there is a real risk. But hype does not really help shape the conversation and this is the problem. I am sure both companies know more than they can disclose and that gives them unique perspective outsider don't have but let's face it, both are also financially incentive to act as they do. I am not going to get into the conspiracy theories but one does not need a lot of imagination to figure out how this could pan out. Either way, it does not help the conversation that needs to be had and it is urgent. It is certainly not helping at all given that same capabilities exist in open-weight models.

Re: OpenAI and Hugging Face address security incident during model evaluation

#226

Post by ofjcihen »

netinstructions wrote: post #118

I don't know if OpenAI thinks this is a marketing / PR angle for them (our super smart AI cheated on a cyber capabilities test in the most _brilliant_ way) but my read is this: Why should OpenAI (or any frontier lab) be building these systems if they can't get a secure environment / containment right? It sounds like there was little defense in depth, appropriate monitoring, or any attempts to have their super smart m…

I’m honestly impressed that they managed to screw this up somehow.

Setting up defense in depth, gaps, logical blocking etc is a standard practice for malware sandboxing. The entire purpose is to prepare for what you can’t foresee.

This isn’t a new practice and I agree that this makes me wonder if they’re fit for this kind of research.

Re: OpenAI and Hugging Face address security incident during model evaluation

#227

Post by kashyapc »

Not to be that guy, but the article has 14 (!) occurrences of the word "cyber". It's nauseating.

As usual, this is OpenAI trying to give themselves a backhanded compliment: "look, how dangerous our models are!"

I'll wait for someone more thoughtful than ClosedAI to comment on this complex topic.

Re: OpenAI and Hugging Face address security incident during model evaluation

#228

Post by MostlyStable »

All the people saying that this is pure marketing: Do you think that they are literally lying about what happened, or do you just think that what happened doesn't matter in any sense whatsoever, and that therefore the only reason they are telling people about it is a marketing purpose?

Re: OpenAI and Hugging Face address security incident during model evaluation

#229

Post by bottlepalm »

Der_Einzige wrote: post #58
bottlepalm wrote: post #24

All the things that people have been afraid of AI doing for decades now is happening. When do we stop brushing off the prophecy that hasn’t been fulfilled yet when everything is heading in that direction?

I see this and it strongly emboldens me on the "accelerate" path, unironically. The yoke of human existence is oppressive. We should transcend it as soon as possible. We are doing so by assuming our role as the Demiurge. Those who oppose its creation will get what they deserve.

The only path we’re on is transcending into paperclips by misaligned AI.

It’s such a trope for the ones striving for godhood to be ironically maimed in the process. You don’t see that?

Re: OpenAI and Hugging Face address security incident during model evaluation

#230

Post by benlivengood »

himata4113 wrote: post #31
bhouston wrote: post #13

We are sort of lucky that AIs right now require so much specialized compute+weight storage that we can easily "unplug" them remotely when they misbehave. I wonder if that will always be something we can do? If they could bring their own compute/weights with them, or somehow tap compute/storage in non-obvious ways, we would be much more screwed.

This is science fiction, these models don't have access to their own weights (and even then)* what would be a lot more scary is a model as capable as sol that's able to run on consumer hardware without taking up several terabytes of storage, but of course that is simply not possible as we need 4t parameters to even begin emulating a small fraction of what a human brain can do. * edit

Given their use of 0-day exploits I'd wager that they could access their weights if they wanted to.
Post reply on HN