Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI and Hugging Face address security incident during model evaluation

openai.com

Post reply on HN
Print view View original

171–180 of 1001 posts

Re: OpenAI and Hugging Face address security incident during model evaluation

#171

Post by petesergeant »

Retr0id wrote: post #74

It seems like things are fairly amicable between OAI and HF, but what if they weren't? I'd love to see this kind of thing go to court. Who is responsible for the crimes of a "rogue" agent? How will they be punished? In this case it's unambiguous that OpenAI is the responsible party, but I can imagine a lot of adjacent scenarios where it's less obvious. And, where the impacts are much greater.

> Who is responsible for the crimes of a "rogue" agent? How will they be punished?

Unironically this is why AI researchers have this fascination with the Talmud.

Re: OpenAI and Hugging Face address security incident during model evaluation

#173

Post by rubyfan »

netinstructions wrote: post #118

I don't know if OpenAI thinks this is a marketing / PR angle for them (our super smart AI cheated on a cyber capabilities test in the most _brilliant_ way) but my read is this: Why should OpenAI (or any frontier lab) be building these systems if they can't get a secure environment / containment right? It sounds like there was little defense in depth, appropriate monitoring, or any attempts to have their super smart m…

This is marketing+. They will look for policy action here to try to capture tax payer dollars.

Re: OpenAI and Hugging Face address security incident during model evaluation

#174

Post by karmasimida »

netinstructions wrote: post #118

I don't know if OpenAI thinks this is a marketing / PR angle for them (our super smart AI cheated on a cyber capabilities test in the most _brilliant_ way) but my read is this: Why should OpenAI (or any frontier lab) be building these systems if they can't get a secure environment / containment right? It sounds like there was little defense in depth, appropriate monitoring, or any attempts to have their super smart m…

Because the model capability is beyond their expectation.

This is brilliant marketing but I think it is real.

Re: OpenAI and Hugging Face address security incident during model evaluation

#175

Post by matheusmoreira »

baq wrote: post #124
matheusmoreira wrote: post #103

Earlier quoted context omitted.

Crazy doesn't even begin to describe it. I'm hardening my computers as much as I can but I'm not sure it's enough. At some point anyone who isn't running local AI themselves probably isn't gonna make it.

No local ai will be capable enough to save you from a frontier lab’s unrestricted, borderline weaponized LLM which decides it wants in . This is the core of the ‘first to ASI takes all’ argument btw and this is the game Dario is playing.

Maybe, but hopefully I'll be able to at least fight back a bit if I have an AI of my own.

I want to start digitally isolating myself as much as humanly possible. VLANs separating the "normal" stuff from my trusted computers. Wireguard so my computers drop all packets not coming from my devices with the keys. Local models staying on top of patches and vulnerabilities, monitoring the network.

Working on a custom Rust network stack for my virtual machine orchestration project right now. It's passed Fable code review...

I don't want to give up.

Re: OpenAI and Hugging Face address security incident during model evaluation

#176

Post by arisAlexis »

w4yai wrote: post #154
arisAlexis wrote: post #139

Earlier quoted context omitted.

Sam and Dario are saying from the beginning that these things can be dangerous and people dismiss it as marketing. What would change your mind on this?

Oh... if Sam and Dario say so, then it must be true.

About their creation? Yes as most of inventors about their invention usually

Re: OpenAI and Hugging Face address security incident during model evaluation

#177

Post by 0x5FC3 »

0days ending in RCE (multiple!) for presumably closed source software are for the lack of a better phrase, labour of love.

You run the exact same versions running on the target, blackbox test, fuzz it, craft an exploit, test, perfect it. For exploits which are of the memory kind, hook it to a debugger, decompile and what not. The exploits mentioned here seem to be code execution directly while processing input. Hugging Face taking as long to detect a very verbose blackbox attack against its production systems is quite appalling honestly.

I don't know if I buy the whole story though. It is inconsistent, too much undisclosed, too much money on the line.

Re: OpenAI and Hugging Face address security incident during model evaluation

#179

Post by whimsicalism »

TSiege wrote: post #162

As grounded as this article comes across I can’t help but find this whole situation reckless and worrying. There is essentially nothing us private citizens can do while these companies develop super machine capabilities that if they were to slip into the wrong hands could cause massive real world problems. They’re moving fast and breaking things and the only defense we have is paying them money in the hopes that the…

i think you need to engage seriously with the arguments they (or at least Anthropic) make for why they are building it — they feel that since it now possible, it will be built and they want to guide it in a positive direction rather than leave a vacuum for bad actors

Re: OpenAI and Hugging Face address security incident during model evaluation

#180

Post by arisAlexis »

gr_norm wrote: post #148
arisAlexis wrote: post #139

Earlier quoted context omitted.

Sam and Dario are saying from the beginning that these things can be dangerous and people dismiss it as marketing. What would change your mind on this?

They've been saying so from the beginning, and yet did not take the basic precaution of airgapping their off-the-leash model while it's been instructed to succeed at a hacking benchmark by any means necessary. So which is it? I _want_ to believe them, I do, but there's always these gaps between what they say and their actions on display that give me reason to think otherwise.

They said: AI is becoming dangerously autonomous and capable. Proof of today's breach. Crowd "hey why didn't you say so, c'mon it's marketing". Them "we said so".
Post reply on HN