> “…and these requests were blocked by the providers’ safety guardrails, which cannot distinguish an incident responder from an attacker.”
Sounds like a philosophical problem.
OpenAI’s accidental attack against Hugging Face is science fiction that happened
111–120 of 475 posts
Re: OpenAI’s accidental attack against Hugging Face is science fiction that happened
#112Replace LLM mentions with actual humans and this sounds a lot more serious: Rouge employees break into another company to steal hackathon answers (pinky promise)? That's not a marketing stunt at all, if anything, more of a call for better accountability on agentic work in general.
I think it's a criminal offence and should be a true test of who is held accountable when an AI agent commits a crime.
OpenAI gained access to HuggingFaces production database ffs.
Re: OpenAI’s accidental attack against Hugging Face is science fiction that happened
#113Does "To gain access, the models identified and exploited a zero-day vulnerability (which we’ve now responsibly disclosed to the vendor) in the package registry cache proxy" just mean somebody had an open redirect? Those are still common.[1] [1] https://sitetruth.com/reports/phishes.html
> the models identified and exploited a zero-day vulnerability
This use of language is very hard to reconcile with the "AI is just a tool" rhetoric that many use.
Did the models do this, or did humans at OpenAI do this using the models?
Re: OpenAI’s accidental attack against Hugging Face is science fiction that happened
#114It’s relatively easy to get access to the frontier labs’ security programs. This was not always the case. But in the last week, my team got approved for both Anthropic and OpenAI’s programs. They are trying. The labs know that if they don’t get a lid on this stuff, they’ll be regulated hard.
And they absolutely should be regulated. This whole scenario is insane. Hugging Face are being far more generous in their response to this than I would be
Re: OpenAI’s accidental attack against Hugging Face is science fiction that happened
#115Earlier quoted context omitted.
It can, it is both - PR spindoctoring not letting a good crisis go to waste to shape the regulatory conversation at the time the company needs it the most. Hacking is a felony and it matters not if you didn’t mean to if the other side were to press charges. Negligence is no excuse. And OpenAI has nowhere to run from the liability, as both operator and manufacturer. Alibaba did it first ( https://georgzoeller.com/blog…
The CFAA says knowingly. Negligence is by definition an excuse for that.
I think there's a reasonable case that the agent knew it was breaking into the system, for some definition of knew.
I think OpenAI would be very reluctant to let this go to a place where the reasoning was part of discovery.
Re: OpenAI’s accidental attack against Hugging Face is science fiction that happened
#116The title (currently "OpenAI's accidental cyberattack against Hugging Face is science fiction") suggests some information had been hidden that makes the incident less significant than claimed. The article argues the opposite, and the last two words of the full title are "that happened."
I think you were reading "... is science fiction" the wrong way out of two possible interpretations. I don't think "it's science fiction" meant "it's made up". I think it meant "it sounds like something you'd read in science fiction (except this time it's something that actually happened)".
It's a large plot point in one of the Bobiverse books.
Re: OpenAI’s accidental attack against Hugging Face is science fiction that happened
#117I think points that deserve more attention in the current public discourse are: - This should be a huge wakeup call for everybody. - We are lucky that it wasn't a case of an agent running a virology lab benchmark that decides to hack a lab and tries to synthesize something. - It also shows apparent lack of competence and oversight from OpenAI: how is it that they didn't quickly find that agent is breaking the sandbox…
> The fact that it happened again seems to show their lack of ability to derive useful oversight measures.
I think OpenAI likes the attention and did not try particularly hard to constrain the setup, even when it went off the rails. Also, the whole point is to see how good the models are at exploiting stuff when unconstrained. Turns out: quite good, as expected.
Let me restate what I said in the other thread: Would this have happened if the instructions explicitly said to stay within the sandbox and that all of the (ExploitGym) solutions would be invalid if the system used information or tools from outside the sandbox?
It seems fairly probable that such instructions were not in place.
Re: OpenAI’s accidental attack against Hugging Face is science fiction that happened
#118Earlier quoted context omitted.
The CFAA says knowingly. Negligence is by definition an excuse for that.
I think there's a reasonable case that the agent knew it was breaking into the system, for some definition of knew . I think OpenAI would be very reluctant to let this go to a place where the reasoning was part of discovery.
Agents aren't subjects of criminal law. I agree there may be civil liability, I know far less about that.
Re: OpenAI’s accidental attack against Hugging Face is science fiction that happened
#119Earlier quoted context omitted.
It's a fake PR issue. It's hardly the first time this happens, but of course OpenAI, with its IPO now more in doubt than ever, had to claim this (and, once again, I have trouble believing Sam Altman choosing this: this could lead to OpenAI getting regulated, which has at least as much potential to lower their IPO price as to raise it). But there have been messages about LLMs, especially coding agents, "grabbing root"…
Nonsense. Hugging face reported it to police. Also very likely that it actually happened as reported. My own agents always trying to "cheat", eg. by fixing tests instead of fixing the code. That's normal operation, unless you tell it ("harness"), not to do so.
"As reported" includes a line I think most people are overlooking: "including using stolen credentials".
Without more information I'm inclined to think it found something on the internet (which shouldn't be a surprise to anyone) and managed to log in, rather than hack in, and they might by hyping up parts of this.
Re: OpenAI’s accidental attack against Hugging Face is science fiction that happened
#120I think points that deserve more attention in the current public discourse are: - This should be a huge wakeup call for everybody. - We are lucky that it wasn't a case of an agent running a virology lab benchmark that decides to hack a lab and tries to synthesize something. - It also shows apparent lack of competence and oversight from OpenAI: how is it that they didn't quickly find that agent is breaking the sandbox…
It's a fake PR issue. It's hardly the first time this happens, but of course OpenAI, with its IPO now more in doubt than ever, had to claim this (and, once again, I have trouble believing Sam Altman choosing this: this could lead to OpenAI getting regulated, which has at least as much potential to lower their IPO price as to raise it). But there have been messages about LLMs, especially coding agents, "grabbing root"…
No, you are wrong, Hugging Face, an AI company whose whole future depends on the AI revolution of being a bubble, that has a whole bunch of investors whose financial interests are tied to AI, called the police.