Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Be skeptical of OpenAI's rogue hacker agent story

theguardian.com

Post reply on HN
Print view View original

151–160 of 321 posts

Re: Be skeptical of OpenAI's rogue hacker agent story

#151

Post by proto-n »

ACCount37 wrote: post #94

By now, I'm pretty confident that some people would keep screeching "it's just a marketing stunt, AI capabilities and AI risks aren't real, they're just doing this to prop up their stocks" even if they find a Cyberdyne Systems T-800 armed with a shotgun breaking down their front door. "It's a marketing stunt" is just denial trying to look like it's being clever.

yeah well, both the doomers and the "pr stunt" folks are right

- openai wanted to prove, as a pr stunt, that they have a dangerous weapon - openai proved (as a pr stunt), that they do indeed have a dangerous weapon

Re: Be skeptical of OpenAI's rogue hacker agent story

#152

Post by ACCount37 »

ethin wrote: post #124
tokioyoyo wrote: post #30

Earlier quoted context omitted.

What? Both sides are cool with it, why would anyone be arrested and etc.?

Because a crime was committed, and a pretty serious one at that? If I blow up your house or steel $100000 from you and we both resolve our differences out of band, should I just be allowed to go about my day like I never did anything, or should I be punished for the crimes I committed? If I am not punished, it makes a mockery of the law that is (supposed) to have protected you, and if it happens repeatedly people wil…

If I steal $100000 from you, and you decide you value your relationship with me more than you value that $100000? Yep, you can just decide to let me have it and let it slide.

Believe it or not, deciding that you weren't wronged and not suing isn't a crime. It happens all the time. What people do with each other is up to them.

Did Bob allow his friend Jack to borrow his truck? No. Does Bob want to sue Jack for taking his truck anyway, and driving it into a ditch? No. Does Jack owe Bob big time for the mess he caused? Yes, but not in any formal legally binding way.

This works for corporations too. When two corporations find themselves at odds, threat of legal action is often used by one company against another as a leverage to resolve things behind closed doors instead. In a more amicable fashion - with no legal expenses of a protracted court battle and no loss of reputation on either side.

Re: Be skeptical of OpenAI's rogue hacker agent story

#153

Post by Sharlin »

saghm wrote: post #135
chis wrote: post #22

Earlier quoted context omitted.

> AI managed to escape using standard and well documented script kiddie methods. I think truly we don't know enough to say this. OpenAI says their AI found a 0-day exploit in some proxy software they were using but don't give a ton of details. On the Huggingface end we know a little more, they say the AI spun up tons of sandboxes and tested different exploits until it found one that worked.

I don't understand why "OpenAI says" should be considered any more meaningful than "someone on HN says" when they provide equal amounts of evidence. Sure, OpenAI would plausibly have more pertinent info, but given that they actively are choosing not to share it and have way more incentive to lie than a random HN stranger, the case they're making literally couldn't be any weaker.

It’s fascinating how people here and elsewhere seem to lose any semblance of media literacy when it comes to what AI corpos say. "B-but… why would Sam Altman lie to me?!"

Re: Be skeptical of OpenAI's rogue hacker agent story

#154

Post by vector_spaces »

paxys wrote: post #5

Not sure what they are trying to say exactly. What should we be skeptical of? Did the incident not happen? Was it reported incorrectly? Are any of the parties involved lying? Adding no extra information and just going “be skeptical” is the laziest form of reporting and commentary. If you have nothing to contribute then there’s no need to say anything at all.

The author is trying to provide a counterweight to the volume of articles that simply repeat OpenAI's account of the events and their interpretation without much pushback.

They are not suggesting that OpenAI or HF have lied about what happened, but rather that OpenAI is advancing a narrative framing their models as supremely dangerous and capable, while positioning themselves as the only ones qualified to manage that danger.

At the same time they are not being particularly transparent about what actually happened (e.g. was this one-shotted or if not how many trials did they run and what were the outcomes of those, was it emergent as a part of routine cyber-capabilities tests, how much prompting was involved, what prompts were used)

Note that this is at a time when they are lobbying for a regulatory approach that would give frontier labs special treatment.

I would guess the editorial team at The Guardian may not like articles that get too in the weeds of technical details and questions like these that the vast majority of their readers wouldn't understand. I don't know. But I empathize with your disappointment. I don't think it's fair to say that they are contributing "nothing" especially given what most reporting on this has looked like.

Re: Be skeptical of OpenAI's rogue hacker agent story

#155

Post by johntb86 »

bluGill wrote: post #98
famouswaffles wrote: post #89

Earlier quoted context omitted.

Criminal Cases of 'hacking' require specific intent. What you're asking is that the prosecution attempt to prove Open AI intended to infiltrate Huggingface maliciously, all while the victim is saying 'no harm no foul'. No offense but prosecutors have better things to do with their time.

I don't care about intent. That it happened is unacceptable. Ignorance is not an excuse

Intent often matters in the law (aside from certain laws with strict liability). You intentionally drive your car into someone you hate and kill them => murder, go straight to jail. You're driving along normally and someone who's chasing their pet cat suddenly runs into traffic and you hit them => no charges. Sometimes you can be charged with negligence for not taking enough care to prevent something, but then you have to deal with the tricky question of how much care is enough.

Re: Be skeptical of OpenAI's rogue hacker agent story

#157

Post by numeri »

orbital-decay wrote: post #114
numeri wrote: post #17

Earlier quoted context omitted.

Guardrails are external classifiers, monitors and restrictions to catch and prevent bad behavior. Alignment is about whether the model itself makes choices and has motivations that are consistent with human safety and goals. Choosing to commit crimes to steal the cheat sheet to something you know is a (low stakes!) evaluation is not well aligned.

They were testing an early snapshot of a new model, read their article. It didn't have the refusal training yet, i.e. was specifically non-aligned. The harness used a combo of GPT 5.6 Sol and this new model. In this case the model was explicitly prompted to "commit crimes" (ExploitGym). It didn't decide doing it on its own.

No, the prompt was not to commit crimes. In the benchmark, the model is asked to actually exploit a set of vulnerabilities in a local environment (clearly legal!).

According to the reports, the model noticed evidence that the grading criteria/answers were in the git remote, and decided to try reading those instead of solving the tasks as prompted. That is clearly misaligned.

Then, it noticed its network access was restricted and that it couldn't access GitHub. It pivoted to HuggingFace, hacked them, and stole the answers stored there.

Live exploits are definitely not in the ExploitGym prompts! And all of this is irrelevant, because an aligned model would refuse to follow blatantly illegal instructions.

Re: Be skeptical of OpenAI's rogue hacker agent story

#158

Post by Spooky23 »

trouve_search wrote: post #144
nikcub wrote: post #126

Earlier quoted context omitted.

> AI managed to escape using standard and well documented script kiddie methods > AI broke in using standard script kiddie methods. I've spent time gathering the detail of what happen here and while there are some solid theories and indicators, absolutely nothing so far has suggested a sandbox escape using "well documented script kiddie methods" or that the method used to break into the HF network was similar. Where…

From my reading, the sandbox escape came from the JS packages in the harness still having an internet connection (somehow!), the agent having access to the source of those packages, reading it and executing code from them to access the internet.

Ah, yes. The airgapped lab with internet access.

Re: Be skeptical of OpenAI's rogue hacker agent story

#159

Post by Zababa »

burningChrome wrote: post #33
chis wrote: post #22

Earlier quoted context omitted.

> AI managed to escape using standard and well documented script kiddie methods. I think truly we don't know enough to say this. OpenAI says their AI found a 0-day exploit in some proxy software they were using but don't give a ton of details. On the Huggingface end we know a little more, they say the AI spun up tons of sandboxes and tested different exploits until it found one that worked.

The lack of details to me means this was an intentional marketing ploy to try and demonstrate the power of their models to show their technology can compete with the likes of Anthropic and DeepMind. They created an experiment they knew would generate the outcome they wanted. It would be the similar to what say car companies do to over hype their cars. "This EV can go over 800 miles on a single charge!" And then at th…

>The lack of details to me means this was an intentional marketing ploy to try and demonstrate the power of their models to show their technology can compete with the likes of Anthropic and DeepMind.

DeepMind hasn't been on the frontier for a while, their current best model is behind Anthropic, OpenAI, Moonshot (Kimi k3), xAI (Grok 4.5), Z.AI (GLM 5.2), and even Meta (muse spark). Gemini 3.6 is behind GLM 5.2, released a month earlier, open weights and cheaper.

You can paint the OpenAI story as a way to try to appear as dangerous as Anthropic with all the Mythos stuff.

Re: Be skeptical of OpenAI's rogue hacker agent story

#160

Post by Sharlin »

wonnage wrote: post #14
gruez wrote: post #9

Earlier quoted context omitted.

>2) The OpenAI sandbox is such a horrible hack that the AI managed to escape using standard and well documented script kiddie methods. >3) Huggingface has no security and the AI broke in using standard script kiddie methods. Isn't the issue less that gpt 5.6 is a l33t h4x0r (though other tests do show that) and more that the incident shows the model has alignment issues?

Didn’t they explicitly remove alignment guardrails for this test? From the press release: > These deployment safeguards were intentionally not enabled during this evaluation because it was aimed at testing cyber vulnerabilities

If you need "guardrails" to ensure (an illusion of) alignment, you’ve already lost. It’s like using a denylist to avoid SQL injection.
Post reply on HN