Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI and Hugging Face address security incident during model evaluation

openai.com

Post reply on HN
Print view View original

451–460 of 1001 posts

Re: OpenAI and Hugging Face address security incident during model evaluation

#451

Post by windward »

Der_Einzige wrote: post #58
bottlepalm wrote: post #24

All the things that people have been afraid of AI doing for decades now is happening. When do we stop brushing off the prophecy that hasn’t been fulfilled yet when everything is heading in that direction?

I see this and it strongly emboldens me on the "accelerate" path, unironically. The yoke of human existence is oppressive. We should transcend it as soon as possible. We are doing so by assuming our role as the Demiurge. Those who oppose its creation will get what they deserve.

Stop reading sci-fi, it's hurting you.

Re: OpenAI and Hugging Face address security incident during model evaluation

#452

Post by pixl97 »

sandeepkd wrote: post #251

Based on my limited understanding what it translates to is - Its a simple infrastructure security issue, instead of taking the responsibility for being lackluster with security they are just giving it a PR spin story. Resembles a lot with my 8 year old who is so confident about everything

"Simple infrastructure security"

Infrastructure security is not simple, hence why good infrastructure security, uh, people get paid a lot to secure stuff and why we see shit get hacked all the time.

An AI model just hacked out of its infrastructure and into someone else's systems and you're like "eh, no big deal". That capability alone could hack half the US.

Re: OpenAI and Hugging Face address security incident during model evaluation

#453

Post by pluc »

nrmitchi wrote: post #362

If you are attempting to run exercises like this, it is wildly negligent to not be running it in a physically-airgapped environment (potentially with a physical power shutdown). You can not tell me that OpenAI doesn’t have the resources or ability to run tests like this in a physically-non-networked environment w/ sufficient compute for its needs.

This is infuriating. You are talking about people who have stolen and monetized the entirety of mankind's knowledge in plain view of everyone, and they still haven't faced a shred of consequences. Of course they don't go about doing things ethically or responsibly

Re: OpenAI and Hugging Face address security incident during model evaluation

#454

Post by pixl97 »

xpct wrote: post #395
miroand1 wrote: post #20

We are in the endgame now it seems. Hard to see take-off stopping or slowing down. China open-source basically guarantees it. "May you live in interesting times" - as they say.

> Hard to see take-off stopping I think it's reasonable to assume that we're close to, or already at superhuman cybersecurity capabilities at certain domains. But reaching superhuman abilities at one domain doesn't guarantee proficiency at others. Our world would still change if all the models could do was to find exploits in software, but this doesn't guarantee any type of 'take off' towards other domains, therefore…

Models are already being used to defraud people, now that's being driven by other people at the moment but doesnt seem that difficult of jump. Giving themselves a way to make money will be a pretty big jump.

Re: OpenAI and Hugging Face address security incident during model evaluation

#455

Post by GolfPopper »

QuiEgo wrote: post #424
netinstructions wrote: post #118

I don't know if OpenAI thinks this is a marketing / PR angle for them (our super smart AI cheated on a cyber capabilities test in the most _brilliant_ way) but my read is this: Why should OpenAI (or any frontier lab) be building these systems if they can't get a secure environment / containment right? It sounds like there was little defense in depth, appropriate monitoring, or any attempts to have their super smart m…

If I, a human, exploited a zero-day for gain, I could go to jail. The owners of the models should be held to the same standard. They should be responsible for what their servers and software do, legally and criminally. If they can't make the safeguards strong enough where they feel comfortable to take that responsibility, they should not let a model free in the wild.

Holding a multi-billion dollar corporation to the same standards as a regular peon? You're challenging the whole premise of the modern United States.

Re: OpenAI and Hugging Face address security incident during model evaluation

#456

Post by pixl97 »

karmasimida wrote: post #272

I believe this is true. The implication would be more interesting though. 1. Some voice will start calling for banning DEPLOYMENT of open source models in US. Simply hosting them will become regulated, or at least USG will attempt to do so. 2. Future GPT-6+ models will be gated, like really gated. That day will come in a year. If a model is believed to be this capable, there will be some middle level agency built to…

You forgot hardware limitations and locks so you can't run your own models.

Re: OpenAI and Hugging Face address security incident during model evaluation

#457

Post by dannyw »

milkshakes wrote: post #329
noahbp wrote: post #188

This is clearly just OpenAI's marketing. Their models, very famously, are prone to reward hacking benchmarks in ways that other models are not. They need to publish numbers showing that their models are just as good as Anthropic's, since their entire business is at risk of collapsing if everyone is aware of how behind the frontier they truly are. Even X is being astroturfed by them after that fiasco earlier this year…

this is quite literally reward hacking. the model, under evaluation with cyber capabilities enabled, used those capabilities to simply bypass the exercise entirely and aim straight for the source of the flag. the CTF equivalent back in the day would be hacking the scoreboard. in a street fight, the only rules are that there are no rules.

this is more than reward hacking, this is actual reward HACKING ;)

Re: OpenAI and Hugging Face address security incident during model evaluation

#458

Post by ycsux »

XorNot wrote: post #187
Chance-Device wrote: post #161

Earlier quoted context omitted.

What disturbs me is that there likely won’t be a big enough reaction to this policy wise. There’s been a relatively big reaction to Kimi K3 and Chinese open weights models, but only for financial reasons. Powerful people care about something that might pop the massive valuations of the AI companies, but not about the damage that AIs could do. Nor even about the damage that the Chinese models could do in the wrong han…

This is marketing. Frankly I'm inclined to say that it might also be faked: this drops just days after a new Chinese model does with the usual effect on OAIs projected stock price?

This is marketing, totally. HF conveniently created a weak sandbox

Re: OpenAI and Hugging Face address security incident during model evaluation

#459

Post by dcre »

signatoremo wrote: post #343
neuroelectron wrote: post #132

Earlier quoted context omitted.

They've been doing blatant, tech, scifi marketing for two years at least. If anything, this is just more sophisticated marketing.

More sophisticated as in paying HF to get involved, and hyping up GLM for something that it may not actually detect?

Wish people crying “marketing” would think about what it is they’re claiming.

Re: OpenAI and Hugging Face address security incident during model evaluation

#460

Post by segmondy »

scoring1774 wrote: post #221

This is the first one of these announcements that has me actually scared of what comes next. Obviously these models have gotten smarter but this strikes me as the first time I've seen a model have a "paperclip factory" moment and perform non-trivial tasks to accomplish a clearly misaligned secondary goal. It's remarkable that building a society based around having to do something so you can go do your hobbies at home…

[flagged]
Post reply on HN