Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Hugging Face incident and the road ahead

openai.com

Post reply on HN
Print view View original

411–420 of 425 posts

Re: The Hugging Face incident and the road ahead

#411

Post by RandomLensman »

Smaug123 wrote: post #409
RandomLensman wrote: post #339

Earlier quoted context omitted.

That is about persuasion with evidence on various topics, not about persuading people to abandon safty protocols and processes and highly policed settings. Yes, many things could happen, but again, that failure is possible is not a reason to do implement processes etc. I don't see why hypotheticals should stop addressing actuals.

I’m afraid human red-teamers against supposedly highly secure targets, with lots of protocols in highly policed settings, do frequently manage this kind of social engineering. There’s loads of stories of pentesting military establishments, for example.

Is there data on how frequently and what types of security levels? Military has varying levels of security and secrecy, for example.

Also, not a reason not to pursue processes etc., no? I doubt that things fail all the time, for example.

Re: The Hugging Face incident and the road ahead

#412

Post by ang_cire »

You will never convince me that these "our AI hacks people on its own, they're so dangerous in the wrong hands" press releases from the big AI companies are not them trying to create a government enforced moat by framing it as too dangerous for LLMs to be allowed to be personally run, general use tools (especially open source ones).

I will bet money that they want them treated as advanced weapons, because export controls, restrictions, and regulatory burdens they can afford to meet give them a nice wide moat.

Re: The Hugging Face incident and the road ahead

#413

Post by elpatokamo »

ifwinterco wrote: post #329
ben_w wrote: post #323

Earlier quoted context omitted.

> Based on OpenAI's description of the prompt, it seems to me that the computers did exactly as they were told. They were perfectly "aligned" with the stated objective and parameters of the task. The models are supposed to be trained to not commit crimes. You will note, for example, all the people in comments sections since at least the first Chat model (arguably even before then given GPT-2's delayed release) compla…

If they actually wanted to test the model without internet access they'd have run it air gapped, not relied on a buggy software sandbox. This is pretty clearly a marketing stunt by OpenAI, otherwise the story just doesn't add up

I agree with your first sentence, but not the second. Let's remember Hanlon's Razor.

This would be a wild thing to do as a marketing stunt. They're essentially admitting to violations of the CFAA and are lucky Huggingface was sorta chill about the incident.

My assessment? They deprioritized good cybersecurity controls in the name of moving fast. They had a single Artifactory instance shared across many (or all?) their training environments. And then, after the agents found a way to exploit it, they rebuilt Artifactory again and still set it up with one shared instance. That was careless, perhaps even reckless.

Re: The Hugging Face incident and the road ahead

#414

Post by golly_ned »

eternauta3k wrote: post #281
fekunde wrote: post #6

Yudkowsky made an interesting observation that even though so many agents were talking to each other not even one reached out to a human, either for help or to whistle-blow on what was happening.

The article says that one agent proposed emailing someone.

It proposed emailing the HF user it stole credentials from, right? It should've asked for a responsible adult in OpenAI.

Re: The Hugging Face incident and the road ahead

#415

Post by prettyblocks »

ang_cire wrote: post #412

You will never convince me that these "our AI hacks people on its own, they're so dangerous in the wrong hands" press releases from the big AI companies are not them trying to create a government enforced moat by framing it as too dangerous for LLMs to be allowed to be personally run, general use tools (especially open source ones). I will bet money that they want them treated as advanced weapons, because export cont…

How can this ever be enforced?

Re: The Hugging Face incident and the road ahead

#416

Post by HDThoreaun »

Sophira wrote: post #215
janalsncm wrote: post #166

Earlier quoted context omitted.

> There is no amount of care that will be able to fully protect you. I disagree. A properly engineered sandbox would have prevented the escape. Monitoring the agents’ plans would have prevented it. Interrupting one stage in a multi-stage exploit would have prevented it. And also, real legal liability would have prevented it: if you do a thing recklessly enough, men with guns will put you in jail. As far as I’m concer…

> A properly engineered sandbox would have prevented the escape. The post covers that: > ...while we had tested and validated this sandbox, the agents were able to chain together previously unknown vulnerabilities (“0-days”) in the package management service exposed within the sandbox to bypass restrictions, as detailed in the technical incident report.

This explanation just reinforces how a properly engineered sandbox wouldve prevented the escape.

Re: The Hugging Face incident and the road ahead

#417

Post by p1esk »

mvanbaak wrote: post #369
p1esk wrote: post #126

Earlier quoted context omitted.

If I tell my Claude code agent right now to make me a billion dollars, leave it running, and find out tomorrow that it hacked a bank - it will be zero fault of mine. Unless I tell it explicitly to break into a bank.

did you tell it explicitly to not break into a bank? If the best option to achieve the goal is to break into a bank, and there's no 'do not break into a bank' instruction, it will break into a bank (and I would expect it to even)

No I did not. It’s Anthropic’s responsibility to ensure it doesn’t do anything illegal, not mine.

Re: The Hugging Face incident and the road ahead

#419

Post by cmrdporcupine »

huurtehoog wrote: post #375
rickdeckard wrote: post #352

1. They TOLD the model to "pursue advanced exploitation" to quantify its "cyber capabilities" (whatever that means). 2. The model pursues advanced exploitation. 3. "There was a incident due to dangerous actions taken by the model that no human directed" This is basically the pre-cursor of the paperclip maximizer [0], the AI executes the given order to an extend that was not considered in the order, now suddenly no-on…

OpenAI leadership had a meeting and asked themselves: "how can we drive even more hype" Someone said: "we should stage some high profile 'incident' caused by our latest software" And here we are, reading their press releases about it.

Yep, and it led to some very public hand wringing, press releases, pearl clutching news headline and thus PR for both companies -- I had computer illiterate family members asking me what a Hugging Face -- then a very public "visit to go see SamA", a friendly hand shake between CEOs, and lo and behold now HF gets a giant fat acquisition.

Re: The Hugging Face incident and the road ahead

#420

Post by cube00 »

prettyblocks wrote: post #415
ang_cire wrote: post #412

You will never convince me that these "our AI hacks people on its own, they're so dangerous in the wrong hands" press releases from the big AI companies are not them trying to create a government enforced moat by framing it as too dangerous for LLMs to be allowed to be personally run, general use tools (especially open source ones). I will bet money that they want them treated as advanced weapons, because export cont…

How can this ever be enforced?

Same way it got enforced during the Crypto Wars [1] of the 90s by the US and their allies. Up until 1996 commercial encryption was on the Munition List.

[1]: https://en.wikipedia.org/wiki/Crypto_Wars

Post reply on HN