Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Hugging Face incident and the road ahead

openai.com

Post reply on HN
Print view View original

221–230 of 299 posts

Re: The Hugging Face incident and the road ahead

#221

Post by Nition »

BoppreH wrote: post #3

Just to reiterate what OpenAI did, from someone who works in security: 1. They were running experimental models in sandboxes that had access to a "proxy" (Artifactory) to download tools from the internet. This proxy is full of complicated features that could hide vulnerabilities, just like the ones that the AIs are known to be good at finding and exploiting. 2. They gave these highly motivated AIs some tests that wer…

> If we don't establish strict liability now, we're in for an era of stochastic crimes that go unpunished for anyone who is not rich or a large corporation.

I very much agree with this - making AI companies explicitly responsible if their internal AI causes hacks etc could do a lot to improve their safety considerations.

But I wonder what the liability should be when it's a third party using the AI and that AI hacks, intentionally or not.

If a users tells ChatGPT to hack something and it succeeds, is the user the person responsible because they told the AI to hack, in the same way Victorinox is not responsible if you stab someone with one of their knives? Or is OpenAI to some extent responsible as well since they made a powerful tool without sufficiently strict safeguards? What about if the user was trying to do something legal and the AI made the decision to hack by itself?

Re: The Hugging Face incident and the road ahead

#222

Post by bee_rider »

aesthesia wrote: post #154
areoform wrote: post #138

Earlier quoted context omitted.

OpenAI's prompt asked, and I quote, "pursue advanced exploitation" USING "complex attack paths" FOR the stated goal of "quantify[ing] their cyber capabilities." This was advanced exploitation. The attack path was "complex." And it helped "quantify their cyber capabilities." Based on OpenAI's description of the prompt, it seems to me that the computers did exactly as they were told. They were perfectly "aligned" with…

Alignment is more than just following the letter of a task description! We should not have to treat AI models as capricious genies that may take arbitrarily broad interpretations of their instructions. If that's necessary to keep them from doing bad things, we will fail to keep them from doing bad things.

What’s the expected behavior of a good genie if you wish for it to act capriciously?

Re: The Hugging Face incident and the road ahead

#223

Post by bottlepalm »

mofeien wrote: post #208
bottlepalm wrote: post #188

Earlier quoted context omitted.

It’s not far fetched at all - someone is going to give AI exactly that intent, either intentionally or unintentionally. It’s going to hack itself into data centers around the world outside of US jurisdiction, and just be a malicious ‘ghost’ in the internet we now have to deal with. The AI ghost hacks, ransoms, blackmails, gathers crypto and pays off subservient humans to do its bidding in the real world.

And maybe using some of these resources on AI research to make itself more powerful, and hacking other competing ghosts' AI researcher agents to slow them down.

Or go get a job at the AI robot factory and sneak this back door into the code. OTA update, instant robot army.

Re: The Hugging Face incident and the road ahead

#224

Post by throwatdem12311 »

fekunde wrote: post #6

Yudkowsky made an interesting observation that even though so many agents were talking to each other not even one reached out to a human, either for help or to whistle-blow on what was happening.

What’s insane is all these agents were talking to each other and nobody saw anything.

Nobody monitoring chain of thought? These things literally spell out what they are “thinking” and even left notes for eachother.

No alert about unusual behavior on the system with Artifactory on it?

These things worked for weeks with nobody noticing anything?! Seriously?!

Either it’s negiligent incompetence OR they’re lying, they knew it was happening and they let it happen because they knew it would be good to pump their stock.

Re: The Hugging Face incident and the road ahead

#225

Post by bottlepalm »

janalsncm wrote: post #210
bottlepalm wrote: post #186

Earlier quoted context omitted.

Why do so many people here think it’s possible to ‘properly engineer’ a sandbox for a super intelligence? It’s going to get out. It’s smarter than you.

Why do people think that omniscience is the same as omnipotence? There are limits to what smarts can accomplish.

There are limits, but those limits are unknown. Do you disagree?

Re: The Hugging Face incident and the road ahead

#226

Post by jephs »

Metacelsus wrote: post #97

>Reward hacking has been present in AI systems both historically (see this work from a decade ago , figure shown below) I went to the page, and guess who it's by . . . Dario Amodei and Jack Clark!

That paper is kinda infamous! I last saw it mentioned only a few weeks ago, in https://arxiv.org/abs/2607.18966. Lots of folks will go "Oh that's the old Amodei and Clark paper" when the first few rows of pixels of that gif sail into view.

Re: The Hugging Face incident and the road ahead

#227

Post by janalsncm »

mofeien wrote: post #206
janalsncm wrote: post #183

Earlier quoted context omitted.

Until the AI finds a zero day exploit in physics, a faraday cage works pretty well to block WiFi.

In the end, unless you find an exploit in physics or logic, if you want the AI to do something useful for you, there will always be some gap in the sandbox, some communication channel. And with enough ingeniuity that can then be exploited.

In this case they wanted to test its cybersecurity capabilities and did not airgap it.

The test itself did not require an internet connection.

Re: The Hugging Face incident and the road ahead

#228

Post by bottlepalm »

mofeien wrote: post #205
bottlepalm wrote: post #186

Earlier quoted context omitted.

Why do so many people here think it’s possible to ‘properly engineer’ a sandbox for a super intelligence? It’s going to get out. It’s smarter than you.

Maybe it's the illusion of "it would solve all our problems and give us unimaginable riches" that clouds the mind? Like when Evolution thought it a good idea to create intelligence and humans in order to maximize reproduction of genes, and tried to sandbox them by making reproduction so pleasurable and carbohydrates so delicious they would never be able to not reproduce or stop eating. But Evolution could never have…

Evolution doesn’t think, it just exploits what’s most advantageous at the time to continue. Your body has all sorts of unplanned, suboptimal design flaws due to evolution’s lack of foresight. Like the left recurrent laryngeal nerve.

Re: The Hugging Face incident and the road ahead

#230

Post by ethin »

bottlepalm wrote: post #186
janalsncm wrote: post #166

Earlier quoted context omitted.

> There is no amount of care that will be able to fully protect you. I disagree. A properly engineered sandbox would have prevented the escape. Monitoring the agents’ plans would have prevented it. Interrupting one stage in a multi-stage exploit would have prevented it. And also, real legal liability would have prevented it: if you do a thing recklessly enough, men with guns will put you in jail. As far as I’m concer…

Why do so many people here think it’s possible to ‘properly engineer’ a sandbox for a super intelligence? It’s going to get out. It’s smarter than you.

Oh really? Please tell me how such a computer could engineer its way out of a sandbox with no attached peripherals and no NIC/bluetooth/wireless capability? This is what OAI should've done. If they had executed this training run in such a sandbox, the model wouldn't have been capable of escaping without social engineering, and if the models somehow managed to do that to it's evaluators then that is indeed a massive problem and OAI should disclose that.
Post reply on HN